Friday, 7 November 2014

Tim’s Vermeer - II. DSD

Yesterday I wrote about Tim Jenison and his cool research on the topic of Vermeer and the photo-realism of the Dutch School, captured in a wonderful documentary called “Tim’s Vermeer” from Sony Classics.  I mentioned the extraordinary story of how Jenison constructed a plausible apparatus by which, he posited, Vermeer may have actually produced his revolutionary paintings.  Jenison, a graphics designer and non-artist, went further and used it to produce his own version of Vermeer’s “The Music Lesson”, something which, had he done it in 1650, might conceivably have elevated the name Jenison into the pantheon of the greats.

Jenison’s technique in effect had him work his way across the canvas, comparing each fragment of the painting with the corresponding fragment of an image of his subject produced by a camera obscura.  In essence, you consider a spot on your canvas, and compare what the obscura image suggests should go there with what is already there.  If you perceive a discrepancy, you can easily correct it.  Or modify it later if, maybe as a result of what you put in an adjacent area, you come up with something better - in contrast to, for example, how an ink-jet printer might approach the same job.  “The Music Lesson” is about two feet square, and it took Jenison something like 130 days to complete the painting.  The unbreaking concentration required was enormous, and in turn nearly broke him.

While watching all this I was immediately struck by an interesting comparison with the Sigma Delta Modulator (SDM) used to produce a DSD data stream.  The SDM takes some input data, which may be an analog signal or a digital data stream, and sets about producing an output data stream.  Each time it needs to create an output value it sets about comparing two things - the input value, plus the input values that preceded it, and the previous output values.  It uses those to calculate what the new output value should be. 

This is like Jenison’s apparatus.  He goes to a place on the canvas, looks what’s there, and compares that to what’s in the equivalent place on the original image.  He then uses his judgement to decide what he needs to paint in that particular place.  In a practical sense, it’s not that he seeks something right or wrong in absolute terms about the smudge of paint that needs to be applied, more a question of what looks best given the available comparables.  “Using his judgement” is a convenient phrase which undervalues the colossal amount of visual processing power that the brain is able to bring to bear on the task.

This illuminates one of the limitations of a true SDM in a DSD application.  DSD requires that the output of the SDM has to be either 1 or 0.  If, on balance, the SDM figures out that the best output value is actually 0.5, DSD doesn’t have that as an option.  It has to choose either 1 or 0.  The SDM architecture only allows us to look historically at both input and output data and use that to make our choice.  If both 1 and 0 are equally wrong, then it does’t matter which one we choose.  We just hope that the SDM can take the error fully into account when it comes to choosing the next output value, and the ones after that.  In fact the situation is always like that.  The SDM, in reality, always figures out a best output value somewhere between 0 and 1, and never comes up with an output value which is either exactly 1 or exactly 0.  And, unlike Jenison, the SDM doesn’t get to go back and do a make-over once it’s made its choice.  In that regard, the SDM is a bit like the ink-jet printer analogy.

Given that the ideal output value is never 1 or 0, and that we have to pick one or the other and hope for the best, what do we do if it turns out that we’d have been better off choosing the other one?  The answer is that, in the grand scheme of things, we end up with a combination of higher background noise and increased distortion.  But in the end, by designing our SDMs optimally, we do get those parameters down to the point where the overall performance is pretty darned good.

Actually, there is a way to get around that problem.  Let’s take an ordinary SDM whose output value is going to be either 1 or 0.  We can do some kind of “what if” calculation, and say “What if we chose a 1?” and calculate what the output value after that was going to be.  We can do the same thing for “What if we chose a 0?”.  In each case the SDM will chose either 1 or 0 for the subsequent output value.  What we are doing is, instead of selecting between two possible output values, 1 and 0, we are selecting between 4 possible output sequences, 10, 11, 00, and 01, which begin with either 0 or 1.  We get to choose which of the 4 gives us the best result, but this comes at the expense of a doubling of the amount of processing that we have to do.  Note that by choosing between those four possible values, we are only selecting the first bit, and not both bits of the sequence.  In other words, if we prefer 11 or 10, then all we are doing is selecting the single output value of 1, and if we prefer 01 or 00 all we are doing is selecting the single output value 0.  This process is called “Look-Ahead” for obvious reasons.

Look-Ahead can give seriously improved performance in both noise floor and distortion, but in order to achieve that, it turns out you need to be able to look a long way ahead, not just two or three bits.  In reality, 10 or 16 bits of look-ahead are required, and, at first sight, each additional bit of look-ahead doubles the amount of processing time required.  At that rate, 16 (or even 10) bits of look-ahead comes at a prohibitive processing cost.  However, like in most things mathematical, when smart people are motivated to look into it, solutions can often be found.  By analyzing how the mathematics of look-ahead works, you realize that the same calculations are being repeated in multiple branches of the look-ahead tree, and can find ways of only doing them once.  Additionally, some of the branches can be identified early on as being bad candidates for the final decision and can be pruned at an early stage.  Finally, it is possible to expand the basis upon which we decide how ‘good’ or ’bad’ a branch is, and thereby do a better job of eliminating the ‘bad’ ones early on.

Taken together, along with the relentless rate of progress of computer power, these “look-ahead” SDM architectures are on the verge of being implementable.  They may not have an immediate impact on consumer DACs, but they could make their presence felt in DSD studio equipment, where the lower noise floor and distortion may open the door to effective mixing and other rudimentary signal processing.

All stuff that Vermeer would never have thought of.  Or, for that matter, Tim Jenison.

Thursday, 6 November 2014

Tim’s Vermeer - I. Tim’s Vermeer

I finally got my AppleTV replaced by Apple.  It’s not that it took them a long time - in fact they replaced it on the spot with no hassle at all - it’s more that it took me a long time to get round to hauling my ass to the Apple Store.  So now I have a brand new AppleTV 3, presumably without the known bugs that afflicted my original unit.  What we now need to find out is what difference that has made to my AirPlay network, given that the latter has performed flawlessly since I removed the AppleTV from it.

But that’s going to have to wait a short while, because there is something else I want to write about first.  You see, as I mentioned in a previous post, my AppleTV’s main role in life is to drive the TV set in my gym, to provide the boredom relief necessary to get me through my daily workout.  So this morning, I watched a documentary from Sony Classics called Tim’s Vermeer.  It was a profoundly interesting program, and I felt it necessary to write about it, and about the parallels I drew from it (which will be the subject of a separate post tomorrow).

Johannes (Jan) Vermeer was one of the Dutch Masters who painted in the latter part of the 17th Century.  His paintings, typical in general of the Dutch Golden Age, possess a quality we today refer to as ‘photo-realism’.  They possess an accuracy of perspective, and of illumination, that we today take for granted in photographs, but which was quite unknown in Vermeer’s time.

Tim Jenison is an American entrepreneur who built a successful career in software for the TV and video industries.  Although he is a graphic artist, he is not trained in any way as a painter.  Jenison, like many people before him, was deeply intrigued by how the Dutch Masters, and Vermeer in particular, were able to make the leap in perception which they did.

As an expert in the field of video, he came to appreciate that Vermeer’s paintings differed from many more modern artworks in a key aspect.  The way he saw it, the works of the the Dutch School looked more to him like video stills than photographs.  This would only come about if they were ‘copied’ from life, rather than created independently where you can continuously modify the result if it isn’t exactly what you want, even to the extent that what you want is no longer a strictly accurate representation.  Many experts in the field have postulated that the Dutch School arose due to the concurrent development of the “camera obscura”.  This would throw an image of a real-life scene onto a screen or wall in a darkened room, and the artist could paint from that.  Vermeer’s “video-still” brand of photo-realism could arise if he painted by ‘copying’ what he saw on a camera obscura image.

Books have been published on this topic (the so-called Hockney-Falco thesis, named for the British artist David Hockney and the American physicist Charles Falco), which made an impression on Jenison.  The interesting thing is that none of Vermeer’s works show any evidence of the sort of procedures an artist would presumably have had to follow if that were the case.  A camera obscura (latin for “Dark Room”) is a low-light environment, and not one at all conducive to painting a masterpiece.  Therefore, if an artist were to use one as the tool to throw an image directly onto a canvas that he would then paint over, it is likely that he would record the basic framework of the image in the camera obscura, and finish it off elsewhere.  However, X-ray analysis of Vermeer’s works show no evidence of any such structures beneath the final layers of paint.  His works appear to have been deposited in their final form directly upon the canvas, and with extraordinary precision in some critical aspects.

Intrigued by these findings, Jenison set about his own experiments, to see what would happen in practice if you tried to paint Vermeer-style art from camera obscura images, and from there to imagine how Vermeer might have responded to these challenges.  One of the obvious problems is that the image in a camera obscura is upside-down and back-to-front.  Although the latter is not too much of a hindrance, the human brain - and the artist’s eye - have a lot more trouble interpreting an inverted image.  Jenson realized that using a mirror would be the simplest way to correct for that problem and set about experimenting with one.

He immediately found an intriguing solution.  He placed a small canvas flat on a table which he positioned directly in front of an inverted photograph.  He then placed a small mirror directly above the canvas, at the same distance from the canvas as from the photograph.  Peering at the canvas from above, a viewer would see the canvas, except for the area where the mirror impinged, where instead he would see a reflection of a small portion of the photograph.  Both photograph and canvas would be in focus.  You could then use the setup as a tool to draw a replica of the photograph, a bit like dividing a picture into a grid of squares like they taught you in high school.

As a graphics designer, Jenison saw this configuration as a 17th century version of an editing window in which you could place the original and the copy side-by-side for comparison purposes.  In particular, it would enable very precise colour matching, which is otherwise rather more challenging than you might imagine, since the human eye/brain combination has very poor absolute colour memory.  Jenson then used this theory to attempt for himself to copy a simple B&W portrait photo using oils.  As a non-painter, this would be his first ever attempt at an oil painting.  You really need to see the film itself to appreciate what an incredible job he was able to do.

Essentially, what his technique does is to nibble away at the whole image, by adjusting his viewpoint so that, bit by bit, the entire image passes by the interface between the mirror and the canvas, allowing him to compare and replicate the exact tint of the applied paint at each point.

Armed with a primitive (but authentic) camera obscura device, his mirror, and his B&W portrait in oil, Jenison visited David Hockney in London, plus a couple of other authorities, to see if there was any interest in the notion that this might have been the technique that Vermeer himself had used.  The reception he received was quite encouraging.  Also, while in London, he was granted special dispensation to visit Buckingham Palace and spend a half hour looking at his personal favourite Vermeer painting, “The Music Lesson”.

He came away convinced of what the next step should be.  He would attempt to use the self same techniques to try and replicate Vermeer’s “The Music Lesson”.  Actually, he would not so much try to replicate the painting itself, rather he would replicate the method by which it was created in the first place.  Given that he was not in any way a painter, and Vermeer is a revered master, this would be a major challenge.

Jenison was quite thorough in his approach.  Like Vermeer, he chose to make his own paints by grinding his own pigments and mixing them with oils.  He made his own furniture when he could not buy authentic originals.  He cast and ground his own lenses to use in the camera obscura.  He recreated as exactly as possible the room in which the original painting was set, the clothes worn by the subjects, the decoration and the furnishings - which included a Viola da Gamba, on which he gave a rustic and rather baroque rendition of the iconic riff from “Smoke On The Water”.

I won’t elaborate on the outcome, save to say that it all comes to a fitting conclusion.  Along the way, a couple of extraordinary things emerge.  One of the first things Jenison observes when he begins his marathon paint job is the appearance of chromatic aberration, caused by the fact that he has obliged himself to use authentic glass and lens designs in his camera obscura.  If he is going to be true to his aim of objective authenticity, he must include the faint blue blurs which are visible at certain high contrast edges.  But, looking at high-magnification images of the original Vermeer, he is astonished to find that it, too, has rendered the same blue blur, in the same places.  There is no reason to believe that Vermeer understood chromatic aberration.

More dramatically, though, part way through the painting, Jenison discovers that his lens also shows some mild pincushioning, a fact that only becomes evident due to the unerring optical accuracy of his method.  Again, and quite astonishingly, the original Vermeer is shown to exhibit the very same pincushion distortion, in such a way as to suggest that not only did he follow Jenison’s method, but also the precise (and highly practical) order in which Jenison implemented it.  Unfortunately, the narrator did not address the question of whether or not this pincushioning had ever been detected by experts prior to Jenison’s work.  That would have been interesting to know.

I found the whole thing to be wonderfully entertaining and informative.  Since the program was produced by Penn and Teller - with Penn Jillet doubling as presenter and narrator, and Teller directing - one can comfortably eliminate the notion that the wool is being pulled over our eyes in the service of a good yarn.  Some limited follow-up research on my part shows that while Jenison’s theory does indeed receive a great deal of credence - seriously unusual in itself for the work of a rank amateur in the rarefied world of fine art - there is little to support it in terms of the historical record.  Vermeer is not known to have had any particular interest in optics, and his personal effects after his death were not found to include a camera obscura or anything similar.

Tomorrow I will explain what any of this has to do with audio.

Tuesday, 28 October 2014

AirPlay Still Good

It has been a week now since I put my AppleTV in its box and took it back to the Apple Store.  Unfortunately, I was told I needed an appointment to receive an audience with a “Genius” in order to get it seen to, and nobody was available.  Since this involves a 45-minute drive through the West Island’s lethal road construction, I haven’t been back yet.

The upside is that I have had a week without an AppleTV in my system, and during that week AirPlay playback has been flawless, provided I followed the (revised for Yosemite) procedure I described last week.  That’s three systems - a 2014 RMPB, a 2013 bare-bones Mac Mini, and a 2009 MBP.  All running Yosemite, all working just fine, first time, every time, with AirPlay.

I thought that was worth reporting.


I now have my new AppleTV from the Apple Store.  To find out what happened when I installed it in the system, read on here.

Thursday, 23 October 2014

AppleTV

I have an AppleTV 3.  It is an incredibly buggy device.  As an audio device it has been a source of frustration for me since day one, to the point where I now no longer use it - ever - as part of my BitPerfect test regimen.  It is relegated to use in my Gym, where I watch YouTube or Netflix with it while working out.

The AppleTV has an annoying habit of dropping its WiFi connection periodically.  Actually, it doesn’t so much drop its connection - it is more like its entire WiFi system shuts off.  This happens after between a few minutes and a few hours of use, and has persisted across several firmware updates.  The solution is to re-boot it.  Sometimes it takes three or four re-boots.  Rarely do I get through a solid hour without it failing.

Today it appears to have given up for good.  I can’t get it to come back up at all.  I have just found out that there is an Apple recall program in place and that my AppleTV is one of the affected units, so it is now boxed up and ready to go back to Apple.  Let's see what happens.

So, while dealing with that, I got round to thinking a little.  If you have read my recent posts on AirPlay, you will have noted that I spent a few days exhaustively testing AirPlay with BitPerfect under Yosemite and iTunes 12.0.1 with mixed results.  I was using both my AirPort Express and the AirPlay receiver in my Classe CP-800 as the target AirPlay device.  It may not have come across in my post, but my test experience seemed to go through two phases.  The first was an initial three-hour phase during which nothing seemed to work at all.  This was followed by a lengthy period during which AirPlay seemed to function with at least some semblance of predictability, as reported in my post, a situation which still persists this morning.

Here is what is going through my mind.  Is it possible that when my AppleTV was active on the network I was having uncontrollable AirPlay problems?  And that as soon as its WiFi transceiver ‘died’ (causing it to drop off the network) things started to play more predictably?  As I write this, it occurs to me that whenever the AppleTV is active on my network, my RMBP seems to want to select it as its ‘default’ AirPlay device whenever it can, even though I never want to use it in that role and therefore never - ever - select it.  Hmmmm….


Read on here....

Tuesday, 21 October 2014

Adventures in AirPlay

I have been working hard on AirPlay to try to understand what it takes to get BitPerfect to work smoothly with it under the combination of Yosemite and iTunes 12.0.1.  Unfortunately I don’t have a definitive answer for you, but I am at least starting to get a handle on its behaviour.  I thought you might be interested to read some of this.

The problem is, it either works or it doesn’t, and I can’t figure out why.  There are two main modes of “doesn’t work”.  One is where BitPerfect’s menu bar icon stays black.  This one happens rarely and generally only at the first attempt.  It means that BitPerfect cannot access the AirPlay Device.  The other is where the icon starts green, goes briefly black, and then stays green but with no music audible.  This means that BitPerfect is streaming music to the AirPlay Device, which as far as BitPerfect is concerned is responding in the way it normally would.  I have been wrestling with every combination of the various settings and sequences that might impact AirPlay behaviour but despite some successes, nothing has proven to be the magic bullet.

My first potential “Aha!” moment was when I got to the point where iTunes would throw up a message to the effect that “I can’t find the Airport Express” and offers me two options, “Cancel” or “Continue using the Computer Speaker”.  The secret seems to be to select “Continue using the Computer Speaker”.  “Cancel” is the wrong choice.  I spent some time trying to determine what would cause this message to appear, but after a while I just stopped seeing it, and I haven’t actually seen it now since early yesterday.  So that remains a puzzle.

The next interesting observation is a significant deviation from the setup that we have been recommending since Mountain Lion and Mavericks.  iTunes has its own little AirPlay icon (next to its volume control) where you can select between the various AirPlay devices and “Computer”.  It used to be that it was necessary to select the desired AirPlay device, but now I am finding that when using BitPerfect, AirPlay never works unless “Computer” is selected, and not the other way around.

Yesterday, using my Mac Mini, it appeared that the required solution was to select AirPlay as the default system output device using Audio Midi Setup, then launch BitPerfect, and have BitPerfect launch iTunes (whether automatically or manually), then select “Computer” as the output device from the iTunes AirPlay control.  But when I came to confirm my findings this morning, I found that it didn’t seem to matter whether or not I set the default system output device to AirPlay or to something else.  All that matters is that I set the iTunes AirPlay control to “Computer”.  You must select the desired AirPlay device (if you have more than one) in Audio Midi Setup.  I even experimented with connecting the Mac Mini to the network by Ethernet (its normal configuration) or by WiFi.  It didn’t make any difference.

While all this was happening, on my RMBP (which had Yosemite and iTunes 12.0.1 installed) it seemed that AirPlay would always work first time.  I have a second, older MBP and so I installed Yosemite and iTunes 12.0.1 on that machine also.  This morning I have added that to the mix.  It seems that both MBPs have no problems at all getting BitPerfect and AirPlay to work together, provided I set the iTunes AirPlay control to “Computer”.  For the most part the Mac Mini also works too.  However, it took three or four attempts, restarting BitPerfect and iTunes each time in between, before it started working consistently.  With each of these Macs, once AirPlay starts working, it seems to stay working until you stop playback for a while, or quit iTunes/BitPerfect.

So there you have a summary of a couple of days of intensive AirPlay experimentation.  Set the iTunes AirPlay control to “Computer” and it will either work or it won’t.  If it doesn’t, then quit BitPerfect and iTunes and start again.  Rinse and repeat as necessary.  You may be lucky in that you have a Mac which is pre-disposed to want to work well with AirPlay (like my two MBPs) or you may be unlucky that your Mac does not prefer to play ball (like my Mac Mini).  It’s all I have at the moment, I’m afraid.  I have no idea whether or not you will see the same behaviour.  I will continue my experiments, albeit at a less intense level, as I am (a) running short of good ideas, and (b) have other things piling up on my plate.


... As an important follow-up, here are some thoughts and observations regarding my AppleTV.

Monday, 20 October 2014

Yosemite / iTunes 12.0.1

We have been working on evaluating BitPerfect on the latest version of Yosemite / iTunes 12.0.1, and we are coming up with a mixed bag of results.  For the most part it is working quite well, but there are two areas of concern for us for the moment.

The first is with AirPlay.  I have two Macs right now that have been updated to the new configuration.  The first is a RMBP and the second is a headless Mac Mini.  I seem to have no problems getting AirPlay to work on the RMBP, but thus far not with the headless Mac Mini.  I have no idea what the problem is.  I am currently updating a second, older MBP, and will see what happens with that one in due course.

The second issue is with the Console App.  We use the Console Log as a valuable debugging tool, but unfortunately, under Yosemite, BitPerfect is flooding the Console with a raft of unhelpful messages.  In effect, this is amounting to a Denial-of-Service attack on the Console App!!  While this seems to have no obvious impact on BitPerfect's performance, it is rendering our primary diagnostic tool almost ineffective.

More on all this as developments arise ....


UPDATE 21 Oct 2014
 

Monday, 6 October 2014

Our Own League Of Nations

One of the useful things about Apple's App Store is that they give you some very detailed breakdowns of product sales, including by Country.  To date, BitPerfect has been sold in 71 different countries, which is pretty amazing when you think about it.  And it was just this week that our first customer from Pakistan joined the BitPerfect community, extending the list now to 72.  [Ask yourself - can you even name 72 Countries off the top of your head?]  So, whoever you are - if you are reading this - I would like to extend a warm welcome to the sole representative of Pakistan to the BitPerfect Community!

If you are interested, here are the 72 Countries:
Japan
USA
UK
Canada
France
Germany
Netherlands
Australia
Italy
Hong Kong
Russia
Switzerland
Sweden
Taiwan
Belgium
Denmark
China
Norway
Singapore
Thailand
Korea
Poland
New Zealand
Spain
Austria
Finland
Mexico
Brazil
Turkey
Greece
Chile
Portugal
Malaysia
South Africa
India
Ireland
Hungary
Czech
Indonesia
Luxembourg
Argentina
Israel
Ukraine
Romania
Croatia
Phillipines
Venezuela
Slovenia
Colombia
Estonia
UAE
Slovakia
Peru
Bulgaria
Uruguay
Lithuania
Belarus
Latvia
Kazakhstan
Macau
Macedonia
Saudi Arabia
Malta
Ecuador
Kuwait
Guatemala
Costa Rica
Dominican
Nicaragua
Namibia
Cyprus
... and ...
Pakistan!