Tip: Upgrading your FLAC collection to FLAC 1.3.1

Music enthusiasts and audiophiles generally view lossy compression as evil. Depending on how it’s done, it can significantly alter the music itself, introducing imperfections and destroying subtle cues. In a world where storage and processing are getting increasingly cheap, the need for lossy compression is constantly reducing. At the present moment, the FLAC compression format is the leader in lossless audio compression due to its well supported, stable, open source nature, and is the format I prefer to have all of my music in.

The world of FLAC has generally been unexciting, with FLAC 1.2.1 being the encoder I’ve used most of the time – released back in 2007. When FLAC 1.3.0 was released in 2013, I didn’t even notice it! This year brings us another incremental improvement – that of FLAC 1.3.1.

An interesting result of having changes in the FLAC encoder is a slight improvement in compression ratio. By having better estimation algorithms, it seems that an improvement of about 1% is achievable if you recompress files made by earlier versions. It might not seem much, but as processing power is cheap, it’s actually quite worthwhile.

Hold on, recompress?

Yes. You heard that right. There’s actually no real caveats I can see to this operation – after all, we are going lossless to lossless which means no losses in quality. From my experimentation, it seems the new FLAC 1.3.1 files are completely backward compatible from a decoding point of view – and if you use the official FLAC binaries to do your recompression, then you gain/lose no metadata either.

I’ve tested files encoded by FLAC 1.3.1 on my FiiO X1 and they play just fine there as well, which is nice. A 1% difference might mean a few more songs on a card, which would be nice.

By doing the recompression, you also test the integrity of your FLAC files, and might see further improvements if they were compressed at a low-compression preset.

Tools Needed

There are many ways you can achieve this if you want it easy – for example, some commercial music conversion software can perform such conversions in batch mode. My main reason for avoiding them is the potential for metadata pollution (e.g. adding “Encoded by Encoder X”) and or damage.

What you will need is to grab the latest version of FLAC. If you’re on Linux, it should be pretty straightforward with your package manager. If you’re on Windows, you’re going to need to download the binaries from the Sourceforge repository.

If you’re on Windows, in order to use the method I outline later to recursively invoke the FLAC encoder, you will also need Cygwin to provide a bash shell and the find utility. Or you can always use a Linux machine.

A slight complication comes about as well, as some files may have been ripped in a way which is not strictly FLAC compliant. These files have their metadata encoded as ID3v1 or ID3v2 tags. Unfortunately the official FLAC tools won’t do any work with such files, so you need to convert your ID3 tags to FLAC tags first. The easiest way to do this is to use Mp3tag on Windows (isn’t it great to be cross-platform?).

Procedure

First, it is advisable to back-up your FLAC files first, in case of any issues.

In order to fix all tag issues, fire up Mp3tag and select your working folder. By traversing your directory of music files, you can sort by tag type. Select all files with ID3 tags, and select the remove tag option. Then, after the tags are all removed, undo the remove tag operation. This will result in the files having FLAC tags written in their place.

Once that’s done, if you’re on Windows, you will need to extract flac.exe from the binary package and place it into your music folder (most convenient), or somewhere you know the path to.

Invoke your terminal – or Cygwin in Windows and change directory to where the files are stored. Then we can begin processing by using the following command:

find -name "*.flac" -exec ./flac  {}  --force --best \;

This command looks for all .flac files in the present directory and recursively into deeper directories, and invokes flac.exe in the current directory, passing the path to the flac file (as in the {}) and chooses to force overwrite the original file with best compression setting.

In Linux (or even Cygwin with the latest flac build, which mine didn’t have), you can omit ./ in front of flac to use the installed flac encoder in the system path, rather than use the local copy of flac encoder in the present folder. Of course, the command itself is very useful and flexible – so if you ever need to invoke a command for every file of a particular criteria … keep this in mind. Unfortunately, I don’t know of an MS-DOS equivalent that’s anywhere near as simple, so hence the need for Cygwin.

Once everything completes, you will be returned to the command prompt. Enjoy your 1% or thereabouts of space savings!

Caveats

Alas, things are not always as rosy as it seems. Sometimes you will find the FLAC encoder will hang at a particular file, because it’s malformatted. This can ruin your day in some way … so note down the file and get rid of it or fix it somewhere. There’s also potentially issues with broken files which cause the encoder to quit halfway.

But don’t worry! The force option does not clobber your input file until the encode succeeds. Instead, you will find some temporary files left over with “tmp” in the name – search for them, investigate the associated FLAC files, and delete them to clean it up.

It is also not a drama to have it screw up mid-run and have to re-run it. It will attempt to recompress already upgraded files, but as it’s a lossless operation, it’s not going to damage anything! It just wastes a bit more time.

The way it’s being invoked in the above example also does not take care of multi-threading, and thus doesn’t make the best use of all the CPUs cores available.

As a result – I would advise that you split the collection of FLACs into folders of about 50Gb at a time, and run several terminal windows at once, with the folders being processed in parallel. In case of issues, you only have to re-do one of the split folders instead of the whole lot. It would also eat up more of the CPU processing power as well, improving the effective throughput of the operation.

Conclusion

If you want to save a tiny bit of space, without losing anything, recompressing your files to FLAC 1.3.1 is probably something that might be of interest. My desire to do this helped me uncover and learn more about the find command syntax, which proves to be a very handy command for file operations, and let me squeeze a little more out of my memory card for the FiiO X1.

About lui_gough

I'm a bit of a nut for electronics, computing, photography, radio, satellite and other technical hobbies. Click for more about me!
This entry was posted in Audio, Computing and tagged , , . Bookmark the permalink.

4 Responses to Tip: Upgrading your FLAC collection to FLAC 1.3.1

  1. nobody says:

    You may want to check out ttaenc (TTA). It’s faster and produces smaller files than LPAC most of the time.

    Best Regards

    • lui_gough says:

      Thanks for that. I have heard of TTA, but generally avoided it due to limited hardware support. Once that changes, I can see it being possibly being more widely adopted. That being said, FLAC does have a dominating position at the moment, and any compression improvements from hereon in are likely to be of even more diminishing returns (I’d say, probably 3-5% more at the best but with hardware decompression penalties).

      – Gough

  2. harney says:

    Fantastic articles Lui loving the new write up on the old skool 10 base net stuff

    H

  3. gduffield says:

    Here’s a much quicker way to accomplish the same thing (change -P4 to -Px where x is the # of CPU cores in your machine:

    find -type f -name \*.flac -print0 | xargs -0 -n1 -P4 flac -f -8 –preserve-modtime –verify –no-padding

Error: Comment is Missing!