Tuesday, 21 January 2014

DSD is coming … yes it is!

We have been talking for a long time about bringing DSD playback capability to BitPerfect, and yet nothing has been forthcoming. What’s going on here? How hard can it be? Time for a little bit of an update.

As long ago as January 2013 at CES in Las Vegas, a special version of BitPerfect was playing DSD in one of the high-end rooms in the Venetian, all controlled through a nifty little iPad App. In fact, Tim spent most of the show sitting in the shadows of the demo room, laptop on lap, coding refinements in close to real time. Anybody who saw him probably assumed he was being consumed by his twitter feed.

Playing DSD is really not at all hard. Once you have the file specification it is just a matter of reading the file, formatting the data the correct way, and sending it off to the DAC. Job done. So how come a year's gone by, and still no DSD support in the product?

One reason lies in the fact that iTunes itself does not support DSD, and so there is no way to import DSD files into iTunes. Below, I will discuss what we have been doing on that front.

Another reason relates to the single biggest drawback of DSD as we have it today. Within the Mac ecosystem, the only way to play DSD is using a format called DoP, which stands for DSD over PCM. I really wish we had been involved in the genesis of this “Standard” because it is fundamentally flawed. The way DoP works is it takes a DSD data stream and dresses it up to look like a 24-bit 176.4kHz PCM data stream, fooling the DAC in the process. Any DAC/Computer interface that supports 24/176.4 PCM will be able to accept this data stream. But although DoP looks like 24/176.4 PCM, if you tried to play it it would sound like white noise.

DoP works by packing the DSD data, 16 bits at a time, into the 16 least significant bits of a 24-bit PCM data field. The remaining 8 bits comprise a special bit pattern. When a DoP-compliant DAC receives 24/176.4 data and sees that each data field has the same special bit pattern in its most significant 8 bits, it knows it is receiving DSD data. It is then the DAC’s job to route that data through its DSD processor, and not its PCM processor. If the data should inadvertently end up in its PCM processor, then the result is white noise at a level of approximately -30dB.

Why, then, is this flawed? Because ultimately it is going to confuse the bejeezus out of everyday consumers. It will confuse them because if they want to use DSD they will have to familiarize themselves with at least the basics of what DoP is about. Consumers are able to deal with the existing bewildering array of PCM formats because they don’t have to understand anything about it if they don’t want to. If you connect your DAC to your computer using USB, the DAC and computer talk to each other and establish the various formats that the other supports. That way, if you try to play a 24/192 track, and the DAC turns out not to support it, the computer knows to convert it to a format the DAC can play, and, generally speaking, does so automatically and without further ado. If the consumer does not know what he is doing, then, in a worst case scenario he is still listening to music, just not necessarily in the format he maybe thinks he is. If he didn't know the difference in the first place, that is unlikely to be all that big of a problem to him. At BitPerfect, we term this fail-safe. If the consumer uses BitPerfect and asks to play a file in a format the DAC does not support, we instead deliver the desired sound in a different format which the DAC does support, and we do that without any further intervention from the customer. And if the customer cares to know, we make sure we tell him what it is we’re doing.

BitPerfect can deliver fail-safe PCM performance because it is able to make a list of every last possible format which the output device can support.  If you look into BitPerfect’s “Device Info Report” you will see all those formats listed, albeit somewhat cryptically.  It can do this because the Mac can communicate with the DAC and receive all that information from it.  But with DSD, if the DAC happens to support DoP, then it actually has no way to report that particular capability to the Mac!  Therefore BitPerfect, in turn, has no way of knowing whether the DAC it is playing through can play DSD files! We have therefore designed the forthcoming version of BitPerfect to have a “Master DSD Enable” checkbox.  Only if you check that box will you have access to the DSD-playback capabilities of BitPerfect.  If you don’t have a DSD-compatible DAC there is absolutely no reason whatsoever to check the check box.

This is why we think DoP is flawed.  It is fundamental problem, and not just a BitPerfect problem.  When any Music Player sends a DoP data stream to a DAC, it has no way of knowing whether the result will be music or white noise.  The Music Player relies entirely on the User taking care to tell it whether or not each connected audio output device supports DSD.  It also relies on the User knowing what he is doing when he makes these settings in the Music Player.   Computer users are generally predisposed to want to toggle any settings they may come across, without the first clue what any of them mean, and then having the temerity to blame someone else because it didn’t do what they wanted it to do.  Reading the manual is rarely accepted as a valid pre-requisite.  DoP is only a great idea if you assume the user has read - and understood - the manual.

There are other inconveniences involved in adding DSD to your music collection. Typically, a User who has a DSD-compatible DAC only has one such item. They tend to be quite expensive, after all. But most computer-based audio setups tend to have an element of whole-house networking associated with them, and they can play music through different DACs in different rooms, sometimes with multiple computers sharing a single database. If you buy an album in DSD format, you can then find yourself in the situation where it will only be playable through one DAC in the whole house. The solution is to make PCM copies of your DSD albums. There are a couple of Apps I know of that will do this. Korg Audiogate is free, but forces you to tweet everything you convert(!!!), and its PCM conversions are of less than stellar quality. Weiss Saracon makes better quality conversions, but is priced to make you whistle hard enough to blow the enamel off your teeth. Fear not, though, BitPerfect has a solution in the works.

Even with high quality PCM conversions, you will still have an awkward situation. Your music library will have both DSD and PCM copies of your music. Everyone in the house therefore will need to know (i) which DACs in the house play DSD, and (ii) which tracks in the library are DSD and which are PCM. And it won’t help all that much to simply identify your albums as “Toad The Wet Sprocket - DSD” and “Toad The Wet Sprocket - PCM”. That voice will still come screaming down the stairs at you - “Dad!!... Which of these albums is it I can’t play in my bedroom?????”.  Here at BitPerfect we think the most obvious solution is to have a single audio file that plays DSD through a DSD DAC, and PCM through a PCM DAC.

Last summer, we thought we had come up with such a file format. It was apparently very elegant, and seemed to work just fine. It stored the DSD pre-converted into DoP format and encoded it in two channels of a multi-channel file, both labelled as “unused”. The PCM version of the file was stored in channels 1 and 2 (normal usage for L and R). This format had to be abandoned, through, when we discovered that some multi-channel receivers would blithely ignore the “unused” flags for the two DoP channels and would proceed to try to play them anyway. This, and other limitations, eventually caused us to abandon the format.

So it was that in the late fall we came up with a better format that we term “Hybrid-DSD”. This format is an ordinary Apple Lossless file format, but the DSD data is hidden cleverly inside it, in a similar manner to the metadata. Being Apple Lossless, it can be imported into iTunes and will play just fine. Any software that supports the Apple Lossless format will be able to read it and play its PCM content, blissfully unaware of the existence of the DSD content. BitPerfect, on the other hand, will recognize the DSD content, and, if the DAC supports it, will play the DSD content, otherwise it will play the PCM content. It is the most user-friendly approach we can think of.

BitPerfect’s implementation of DSD is based on the use of Hybrid-DSD files, since no other DSD file format can be imported into iTunes. But, as discussed above, it is also based on the need for Users to exercise a degree of intelligence, not to mention diligence, in setting up BitPerfect so that only those DACs which are known to support DSD are designated as supporting it. Also, they need to understand such arcane details such as whether the DAC supports the different variants of DSD - DSD64 and DSD128. Thankfully, in the current climate, all those who have gone to the trouble and expense of buying a DSD-compatible DAC, and downloading DSD source material, should have little or no trouble with any of this.

The problem BitPerfect will face is with a certain group of Users - you know who you are, but you’re probably not even reading this - who have no clue what DSD is, but will set about enabling it anyway with the full expectation of suddenly being able to play DSD files. And no, these Users have most assuredly NOT read the user manual. These people are the bane of software developers everywhere.

So, right now, we are entering the final stretch of subjecting BitPerfect to every possible line of misuse that we can think of. While playing DSD, for example, BitPerfect absolutely cannot be allowed to perform volume control, sample rate conversion, mixing, dithering, or any other conceivable sort of signal processing. Anything of that nature will turn the DoP data stream from music to white noise. So, we must check on every possible change of state of BitPerfect, and ask whether or not the right checks and balances are in place to prevent something unacceptable from happening. This is more complicated by an order of magnitude than any other feature that we have ever added to BitPerfect. Already this has had significant ramifications. One aspect of our audio engine turned out to be all but incompatible with some of these possible scenarios. We have therefore been forced to accelerate the launch of our next-generation audio engine, which was originally scheduled to be released following the DSD-support release. DSD support will therefore now be launched in conjunction with the new audio engine. At the moment, it is actually the fine details of the new audio engine, rather than the DSD support, which is undergoing last minute debugging. We will be launching this as BitPerfect Version 2.0.

We thank you for your patience, and we hope you will find it all worthwhile when we finally get V2.0 into the App Store. When it does, I hope you will all take the opportunity to rush out to your local high-end audio store and audition some DSD-capable playback hardware. I am sure there will be very few of you who are not blown away by what DSD can do. These may still be the days of the early adopters, but it is good to at least get a feel for what you may be missing out on.


UPDATE:  BitPerfect 2.0 and DSD Master have now been released.  Read about it here.