Around three weeks ago, on a Friday, I was feeling a little bored at home, so I decided to take advantage of the good weather and head out to do some experimental time-lapse photography. After all, I did have a “barely used” Sevenoak SK-EBH01 to play with, and I have been working around a basic Lightroom + ffmpeg + Handbrake workflow as in my previous timelapses. A timelapse thrives on movement and busyness, so where else would be better than to try it at Circular Quay, the central “ferry terminal” of Sydney. Of course, there is always more to learn, so I suppose I should share some of my experiences.
Shooting
The process of shooting a timelapse is usually rather straightforward. I carry out my Nikon D3200, with my lens of choice, a fully charged EN-EL14a “high capacity” battery, a blank 128Gb SDXC card, an intervalometer, a tripod and the rotating head. Once on site, take a walk around, scope out the surroundings, and decide on a good location, set-up and let it go.
This time around, I wanted to try something a little different – all of my time lapses to date were all done using the Samyang 8mm f/3.5 CS fisheye lens, so I wanted to see what it would be like doing it with a rectilinear lens instead. As a result, this time around, I used my Tamron 17-50mm f/2.8 lens, at 17mm instead.
Another difference was that I wanted to capture a full 360 degree time lapse (as I hadn’t done that yet with this rotating head) and exceed that if possible. I wanted to see how practical it was to “baby” the unit, set it to 180 degrees over 1 hour, and click it on start (as the unit has only a limited selection of rotation times and angles) once it finishes, so I can capture more action. In all, I was expecting to be there for about two and a half hours – making this the longest time lapse I’ve captured so far.
The idea to take some time to film a timelapse actually hit me on Monday, when I visited Milsons Point, only to find that the vantage point was spoilt by fencing and signage which precluded a nice wide field of view. From there, it seemed that Circular Quay was the best side to be on. When I got to Circular Quay, I decided to survey the MCA side of the foreshore, walking all the way to the bridge itself. Unfortunately, at the most advantageous positions, it seems the fencing was even taller and more insurmountable.
In the end, I settled on a position at the end of the public walkway near the Overseas Passenger Terminal, directly opposite the Opera House. Being right at the end of the unfenced area would keep my camera away from fences and people, make it more obvious to people, and give me the widest field of view. I stood around, waiting considerately for fellow tourists and photographers to finish up their shots, before hopping in. After a few trial shots, I pressed the start button on the rotating head and intervalometer, and settled in for a long wait – entertaining myself on my smartphone and having a chat on the phone as well.
Unfortunately, when you choose a good photography position, others want in too, and some people with more guts than me were willing to risk walking on a thin ledge to clamber in front of the camera to get a shot. I would have preferred if they were a little more considerate of my setup, because upon reviewing the data, it seems that someone had bumped the tripod early on, spoiling most of the two and a half hour shoot. How very annoying, but that’s a very “tourist” thing to do, I suppose.
Aside from that, you also get hassled. I thought I knew better, so I tend to choose outdoor locations, in public areas, where there is generally no rules against photography. Unfortunately that wasn’t enough. A random person approached me in a confrontational tone, demanding to know if they had been photographed and requesting deletion of their image. I was caught off guard – since it had never happened to me before, and I generally do my best to be considerate and “obvious” in my shooting. After confirming their location and the fact they would have been photographed, I calmly explained that it was a time-lapse project that had taken two hours to compile at the stage the person had intruded into my shot and that deletion during a shoot is technically difficult to achieve. The person then changed tact slightly, begrudgingly agreeing that it was too much of a demand that I spoil my work, but at the same time alluding that I was likely to be a pedophile photographing children at a beach and complaining that I needed a sign or something obvious as a tripod, with a flashing rotator unit and a “noisy” shutter wasn’t enough. In the name of harmony, I agreed to consider this in the future … but it is unlikely to have solved anything as they would just complain that the signs were too small, blown away, or they just plain “didn’t notice it” and somehow that would be my fault.
Once the set had been finished, the D3200 impressively took a total of over 5740 (!!) RAW shots (just 95 shy of filling the card) on just one EN-EL14a battery and one 128Gb SDXC card. I had a large haul of data to carry home.
The Rules
One of the outcomes of the little scuffle was to actually find out what the rules surrounding photography around Australia is. As it turns out, as much as I thought I actually knew the rules, there are always surprises.
As far as things go, in Australia, people can have no expectation of privacy in a public place, and photography is generally of no issue provided it is done in a public area without prying into areas where reasonable expectations of privacy can be had (i.e. through windows). Complying with requests for deletion are generally not required. It seems that things only get problematic where there are prominently identifiable people and commercial interests are involved, where commercial interests mean the usage of a photograph to promote a product, or where there are photographs of copyright-protected monuments, although these are not always clearly denoted.
Of course, there are always misunderstandings in regards to public areas, with many public areas turning out to be not public areas at all. This includes supermarkets, shopping centres, markets, train stations and interestingly, also includes exceptions for the Sydney Harbour Foreshore Area, Olympic Park and Luna Park which I wasn’t aware of. In the case of Circular Quay, it is part of the Sydney Harbour Foreshore rules, which seems to require the application for a permit for shooting for commercial purposes. As I wasn’t shooting for commercial purposes, I wasn’t covered by this, and looking at the permit application process, they were not going to bother issuing a permit for those with a crew under 10 anyway.
In all, it seems that I was clear of any wrong-doing from what I can tell, but it is disappointing to note that there are murky rules surrounding photography that need to be checked thoroughly, especially when travelling in unfamiliar areas. Sometimes it can feel like an attack on the art and hobby of photography, and it can make it much more tiresome exercise. It seems that many times, the rules are written but not strictly enforced or enforced inconsistently or when it’s convenient. I’ll have to keep that in mind in the future – after all, a tripod is generally a trouble magnet, but it’s an essential part of making a timelapse.
Processing
As with most of my other time lapses, the processing workflow involved downloading all the images and processing them first through Lightroom, to maximise the dynamic range, crop and resize, lens correct and remove chromatic aberration. As this was a set of about 110Gb of 24.2MP RAW images, it took more than 24 hours to actually export them to TIFF, with the right sequenced file names and watermarks embedded. I then used another image management program to convert them to BMP so that ffmpeg could assemble it into an .avi, losslessly compressed in FFV1, to then be pulled into Handbrake for the final encode using x264.
However, this time, I decided we would take on an additional challenge. Seeing as one person objected to being in the time-lapse, I decided to do some work to remove that person. This involved looking for the frames containing that person (localized in time) and then trying to “punch” them out by using earlier frames to compensate for missing data. I decided to do this in Photoshop, primarily using the frame just prior to intrusion, and the frame just past the person leaving the frame.
Sadly, things didn’t turn out as good as I expected – and partly because of my amateurish Photoshop skills. Namely, the first issue is that of object alignment – manual alignment can cause jittering, and as my rotating time-lapse system wasn’t rotating around the hyperfocal point, you get parallax errors as well between shots. The second was slight variations in exposure, which results in a “ghost” person appearing, which is probably due to changes in the sun. Another issue is that the sun moves, so the shadows might not align, and that in the areas where the person was “punched out”, the background remains static, which is relatively obvious in a time-lapse video scenario.
It also was quite time consuming. In all, I find it very hard to believe the person was genuine about their desire not to be photographed, as the person remained in the frame for a full 40-shots, representing 80 seconds of real time, shooting the area with their camera phone, including my set-up. If you’re afraid you might have been captured, and it was your primary concern, you would take steps to remove yourself from the frame immediately especially after seeing the equipment take a shot of you. In the hours it took me to punch her out, I felt a sense of betrayal, and even though I didn’t have to remove the person, their removal turns them into a “ghost feature” as well.
The final “raw” product of the basic workflow looks as following (4k resolution available):
The Issue of Flicker
In the past, the assembled timelapses have had imperfections, but I never really worried. This timelapse, more than the others, had quite a few imperfections – and not just getting the tripod bumped. Watching it carefully, especially towards the end, the sky can be seen to “strobe” quite violently. This fast-changing brightness is flicker, and despite setting manual focus, a fixed shutter speed and aperture, can come about because of:
- Mechanical shutter instability – the timing of the shutter may vary especially when the shutter ages, or due to changes in environmental and operating conditions.
- Mechanical aperture diaphragm instability – mechanical tolerances may mean that electronically actuated diaphragms might not settle in the right size every time.
Because it is intrinsically mechanically related issue, it seems eliminating it entirely in shooting is not necessarily possible. However, my other time lapses to date have had less of an issue, because it used the Samyang 8mm fisheye with a manual aperture ring, which eliminates one of the issues. The other thing seems to be that the flicker gets worse towards the end – this may be attributed to the shutter in my D3200 ageing (it’s over 88,000 actuations), warming up in the sun, or more likely due to shutter variation when the battery is low. So maybe it’s a good idea to shoot less, and not to shoot till the battery dies.
But of course, all is not lost. Some help might be found in post-processing. A few paid-for products are available, with some of them working at the RAW metadata level. Others are available for free, but most of them were designed for timelapses where we have a fixed point of view, by mathematically looking at pixel values. This wouldn’t work very well in a rotating timelapse. Could there be approaches which work well with rotating timelapses?
Another approach is to process the video frames themselves. This isn’t an optimal solution, as a lot of metadata is lost, and further “tweaking” of the limited 8-bits per channel result is unlikely to result in a high quality result. However, for beginners like myself, we always gravitate towards the free options first. It never hurts to try.
Veterans of video editing on the Windows platform will be well aware of Avery Lee’s Virtualdub. This is a basic but powerful video editing software for manipulating AVI files, and can work with a variety of .vdf plugin files for filtering and effects. It has many different options, which are useful for those looking to maximise quality in their editing workflow. Of the plugins available for Virtualdub which can be used for correcting flicker are Donald Graft’s Deflicker and MSU Deflicker.
Where are Donald Graft’s VirtualDub Filters?
A while back, Donald Graft of http://neuron2.net was put in charge of cataloging Virtualdub filters, and even to this date, his site is still mentioned by Virtualdub.org as being the main source of filters. If you try to visit it today, you will see that it’s been taken over by a domain parking mob – no longer existing.
It seems many people have had difficulty finding the filters as well, with many requests for working download links.
Luckily, as most of the tilers were relatively small ZIP files, and the site was mostly static, the US Archive’s Wayback Machine has a working mirror of virutally all of the filters that were hosted there. You can find the page, but the sidebar navigation is broken in the copy, so here is a direct link to the archived Filters page.
A VfW Hurdle
Utilizing my regular ffmpeg workflow, but instead using .bmp files converted from .tiffs, I now render to FFV1 encoded lossless files (i.e. -c:v ffv1) for further processing, only departing from FFV1 in the final encode stage rather than work with JPEGs as MJPEG. This has the advantage of allowing you to perform multiple passes of modifications maintaining the highest fidelity possible – especially since the images were exported from Lightroom in TIFF format.
Sadly, it seems that the VfW compatible FFV1 decoder in ffdshow just doesn’t work with the latest FFV1 encodings. As a result, when in Virtualdub, you end up seeing an error (VideoSourceAVI error: The operation is not supported. (error code -1)).
As it turns out, you have to encode as a “level 1” FFV1 stream for it to work with ffdshow. This can be achieved by changing how you use ffmpeg, by adding the -level 1 argument after declaring the video codec like as follows:
ffmpeg -i input.avi -c:v ffv1 -level 1 -c:a copy output.avi
Of course, this can be done to an existing FFV1 AVI file which was encoded with the latest version in a lossless fashion. The resulting file now loads and processes properly with Virtualdub.
Donald Graft’s Deflicker
Installing Virtualdub filters is as simple as unzipping the .vdf file into the plugins directory inside your Virtualdub folder. Then it’s a case of loading Virtualdub, loading your source file, configuring your filter chain, output options and then saving the output.
This deflicker filter seems to be designed for filtering flicker from recorded film projections which is periodic, and some of the options are likely to cause degradation in the sharpness of the output. Donald Graft’s Deflicker filer doesn’t have many settings to configure, however, as it utilizes a windowing algorithm, it should be able to cope with non-periodic flicker as well.
In my case, I increased the Window size, while disabling Softening and Scene change detection. The output looks like this:
The effects are subtle and not immediately obvious – so I decided to make a side-by-side comparison video with the original source RAW footage on the left, and the deflickered output on the right. This was done by using ffmpeg, again, with a filter complex defined as follows (split into multiple lines for legibility):
ffmpeg -i left.avi -i right.avi -filter_complex "nullsrc=size=3840x2160:r=30 [bg]; [0:v] setpts=PTS-STARTPTS,crop=w=1920:h=2160:x=0:y=0 [lt]; [1:v] setpts=PTS-STARTPTS,crop=w=1920:h=2160:x=1920:y=0 [rt]; [bg][lt] overlay=shortest=1:format=rgb [bgl]; [bgl][rt] overlay=shortest=1:x=1920:format=rgb" -c:v ffv1 -c:a copy output.avi
Sadly, I don’t think the deflickered footage is preferable. While it hasn’t produced significantly adverse output, it hasn’t been able to remove the flicker and instead introduces new artifacts where intentional changes in brightness due to people moving in and out and the changing scene causes the image brightness to “pump” similar to how a microphone AGC distorts the audio signal in the presence of fast changing volume levels. Some of the early subtle flicker was removed, although the strong strobing towards the end still persists.
MSU Deflicker
Another commonly used flicker filter is from MSU.
MSU Graphics&Media Lab, Video Group, MSU filters and codecs http://www.compression.ru/video/
This filter is a little more advanced and claims to be better at retaining intentional changes in brightness from the source footage. I have decided to disable blending, to prevent loss of sharpness.
The result of MSU deflicker is a little different. I had initial problems getting 4k footage to process correctly with strange patterns appearing in the output. After a restart, it seemed to work correctly, but the output contrast can be seen to be compromised already.
A side by side comparison can be seen below.
Sadly, I can’t say I prefer this version over the original either – the loss of contrast detracts from the aesthetics of the original, and the flicker is still present in some areas, with the “AGC” like effect being visible especially where people are involved. As expected, a post-processing filter isn’t exactly an oracle – it doesn’t know what is intentional changes in brightness, and what isn’t, so motion timelapse deflickering in post processing is a difficult challenge which may never be perfected. As a result, it is preferable to reduce any flickering in the capture process and live with whatever the output may be.
Conclusion
This exercise was a rather interesting expedition in itself, and resulted in several key takeaways:
- Check the rules before you photograph, if you can actually understand them, and then realize that many of the rules aren’t actively enforced unless you’re posing problems.
- Some people will try to hassle you just because they can, and sadly, there’s not much we can do about them. But avoiding a protracted confrontation is always desirable, so if it’s not too much, you should just give into their demands if it’s not too much.
- Flicker from mechanical sources in shooting time-lapses with DSLRs seems to be unavoidable, but can be minimised especially with manual lenses with fixed apertures. It seems also desirable not to let your battery run down to empty, especially if your camera’s shutter performs inconsistently under those conditions. The battery did last a surprisingly long time though …
- Perhaps shooting with a shutterless ILC camera is a better idea.
- Donald Graft’s filters are still available from the archives kept by archive.org.
- Deflicker filters are not oracles that can separate intentional flicker from unintentional flicker, and neither can they correct for more severe non-periodic flicker, and instead can produce their own artifacts. Whether this is preferable or not will depend on the scene being shot.
- Rotating time lapses pose additional challenges as it can introduce intentional flicker into the scene.
- ffmpeg produces FFV1 encoded .avi files with a newer encoding that ffdshow’s VfW filter cannot understand – so in case of problems opening FFV1 files in Virtualdub, it may be needed to transcode from FFV1 to FFV1 (level 1).
Every day is another learning experience.
