Gapless Playback problems

For those who are interested, here is a lengthy discussion of how BitPerfect handles gapless playback, and the potential problems which can arise using our approach.

In many music albums, the music flows continuously from one track to the next, and there is no effective point at which one track stops and the next one starts.  This occurs very commonly with Classical Music, but is also found in other genres, such as with live "In Concert" albums.  It is therefore very important for a lot of listeners that gapless playback is managed seamlessly if the music is to be properly enjoyed.

BitPerfect manages playback by pre-loading the music into special memory buffers in the RAM.  The contents of these buffers can be instantaneously moved from the RAM buffer to the Audio Output Device's output buffer.  This "memory playback" paradigm is core to BitPerfect's strategy to maximize sound quality.  It is often the case that a music file is too large to fit in its entirety into one of these buffers, so we create a second buffer, and the next part of the music file is pre-loaded into this second buffer.  When the music in the first buffer has all been played, playback instantaneously switches to the second buffer.  This switch takes less than one microsecond, which in the timescales of music playback is tantamount to zero time.  For playback of long files, the playback engine is regularly switching back and forth between the two buffers.  This is, by a clear margin, the most accurate and efficient way of managing music playback.  [NB:  Actually, we now use Triple-Buffering, a similar system which offers some additional advantages in certain circumstances.]

In order to achieve gapless playback, as soon as playback switches to the last buffer of data from the currently playing track, BitPerfect's strategy is to pre-load the next track in sequence into the unused buffer.  The idea is that as soon as the current track finishes, the next track will be able to play back instantaneously, as though it had been an integral part of the previous track.  Fundamentally, there is no more accurate way to achieve gapless playback.

BitPerfect works with iTunes by communicating with it via Apple's inter-process communication protocols "Apple Scripting Bridge" and "AppleScript".  Using these protocols, BitPerfect can request information from iTunes.  For example, when iTunes wants to play a track, BitPerfect asks it for the name of the file that iTunes wants to play, and then loads that file and takes over playback itself.  BitPerfect does a lot of other stuff too, but that is the essence of it, and it works.  For gapless playback, BitPerfect also asks iTunes for the name of the next file in the playlist, so it can be pre-loaded into the RAM buffer.  This ought to be straightforward, but it does not turn out that way.

What happens is that iTunes will, as often as not, provide the name of the wrong file.  When that happens, at the end of the current track BitPerfect starts to gaplessly play the track which has been pre-loaded into the RAM buffer.  It then checks in with iTunes, and finds that iTunes is actually trying to play a different track, the next one in the actual playlist.  BitPerfect then abandons playing the wrong track, and starts playing the correct one.  How long this takes depends partly on the setting of the "iTunes Response Speed" slider in the iTunes section of BitPerfect's Preferences menu (among other things).

So why does iTunes give BitPerfect the wrong information?  And what needs to be done to persuade iTunes to provide the correct information?  These are questions we would love to be able to answer.  And, thus far at least, Apple refuses to acknowledge that it happens at all, so we cannot even get it registered as a bug with Apple.

We have done a lot of research to analyze what is going on.  For example, we have noticed that iTunes is very consistent in how it provides the wrong information.  iTunes appears to have some sort of separate internal playlist that it refers to.  In this internal playlist, the tracks are stored in a certain order, and when a particular track is playing, iTunes always identifies the next track in this internal playlist as the next track to be queued for playback.  But when it comes time for iTunes to actually play the music, it of course follows the actual playlist which may be totally different.  The internal playlist (or something very close to it) can actually be inspected.  Here's how.  Go into the main Music Library in iTunes and select the "My Songs" view.  If you order this list by "Date Added" (newest at the top) then what you get is the secret internal playlist.  [NB: There are subtle differences in how Date Added is implemented depending on the version of iTunes - albums imported by older versions of iTunes may have their tracks appear in a slightly different order].

If, for example, you purchase a downloaded album, you would typically ask iTunes to import the album by running the dialog "File|Add to Library...".  You would then point the Finder window at the directory containing the album, and click on "Open".  If you do that, then you will find that the tracks are usually imported in track order.  You can check this by listing the files in the "Music" library in order of "Date Added" (newest at the top).  However, this is only true if the "Artist" field (as distinct from the "Album Artist" field) is identical for each track.  If it is not, then the tracks will be listed in alphabetical order of their "Artist" fields, rather than in track order.  This holds true for the current version of iTunes (12.1) but not necessarily for older versions, and may change for newer versions.  The result is that albums whose tracks do not all have the same data in their "Artist" fields may exhibit problems with Gapless Playback.  The only workaround that we aware of is to manually import the album, one track at a time, in track order.  This can be a real pain in the keister, but it is most efficiently done by dragging-and-dropping from a Finder window.
NotePreviously, this post suggested importing manually in reverse track order.  We no longer recommend that.

iTunes has a mysterious internal setting called "Fixed Indexing" which can be toggled on and off by BitPerfect.  What does this do?  Apple talks about there being a "Sort Order" and a "Play Order", and observes that "Fixed Indexing" toggles between the two.  It is unclear from Apple's documentation exactly what any of this means, but it appears to correlate to a limited degree with gapless playback problems.  I tend to keep the "Fixed Indexing" setting checked, and it seems to work for me although I have really no idea why or how.

I should also mention that since version 11.x iTunes maintains an "Up Next" feature.  Whenever iTunes is playing, you can click on the Up Next symbol and get a list of what is in the upcoming play queue.  You can also insert tracks into this Up Next queue and move individual tracks up and down the queue on-the-fly.  It's all very sensible.  All of BitPerfect's gapless playback problems would go away if iTunes would allow BitPerfect to read the Up Next list (i.e. make it "scriptable" in Apple-speak).  But sadly they don't.

Finally, I want to point out that since about version 10.6 of iTunes there has been an additional bug, with a related effect.   Before queuing the next track, BitPerfect queries iTunes to find out if it is in Shuffle mode, and if it is, it does not attempt to queue the next track in the playlist because shuffled tracks should never need to be played gaplessly.  However, ever since version 10.7, iTunes always responds that that Shuffle Mode is off, regardless of its actual state.  BitPerfect, therefore, will go ahead and queue the next track in iTunes' "internal list" and not the next track in the (shuffled) playlist - the two will always be different, which will cause a snippet of playback of the wrong track.  BitPerfect has a check box to disable Gapless Playback, which can be used if you are playing in iTunes' Shuffled Mode (or in any other circumstance where you do not want your music to be played back gaplessly).

Sorry for the long post, but a lot of people have been asking for a detailed explanation, and this is it.