Nathan Mhyrvold, Penguins and Digital Photography

BoingBoing has a nice link today to Nathan Mhyrvold's penguin photos from the Falkland Islands.

In June, Mhyrvold wrote a fantastic piece on the future of photography in the New York Times which I encourage all technologists and digital photographers read.

I'm eagerly awaiting Canon's next move, probably to 25-plus megapixels. I'm what marketing people call an early adopter, but mark my words - you'll own a 16- or even a 25-megapixel point-and-shoot in a few years, and it will not stop there. By some estimates, your eyes have an effective resolution of more than 500 megapixels. If you can see it, why shouldn't a camera record it? The reason many pictures don't turn out is that in daytime the human eye can easily perceive a dynamic range of 10,000:1, while at night it is more like 1,000,000:1. Meanwhile, color slide film can record only about 32:1, and digital cameras, about 64:1.

In many situations, this forces a choice - do you expose for the light parts of the scene and let the dark parts go dead black, or save the shadows and turn the bright parts pure white? Future digital sensors will fix this, with ever broader dynamic range and greater light sensitivity (the ISO rating).

Focus is another problem. How many of your pictures wind up fuzzy? Autofocus technology can help, but cameras today still have a limitation on how much of a scene can be in focus at one time, known as depth of field. If you focus on the flower in front of you, the mountain in the background is apt to be fuzzy. Yet technically there is no reason we can't get essentially infinite depth of field, again by using more digital processing.

You can find my penguin photos from Antarctica here and here, taken with an early digital camera, a Nikon E5700 in 2003.

Interestingly, Mhyrvold was also the VP at Microsoft that hired me into the research group there in 1991. I began my work there on Karen Hargrove's, Microsoft At Work.

Technorati Tags: antarctica, digital cameras, falkland islands, mhyrvold, penguins, photography

Update:

My neighbor emailed this photo as a reminder that the artists will always want to control depth-of-field. What will that feature be called? Old-school Focus?

"One comment about your post, though: lack of depth-of-field is not necessarily a limitation. How often do you really want everything in focus?"

Comments

[johnny chunders]

Megapixels are one thing, but image quality always comes down to whatever the light passes through before the projected image circle gets recorded.

Megapixels will not matter past a certain point, because the photographic image is always at the mercy of the lens collecting it. Past a certain point, megapixels won't matter, because additional pixels will only allow the image to gain much more data of the same image resolution.

I think that if anything, the sensors themselves will get bigger to collect a larger image to begin with (over more surface area), before a little snapshot camera will capture 30 megapixels.

Also, we're at the point where the current technology in digital photography can produce increasingly-convincing digital prints. True, c-prints and silver prints still look different than digital prints, but they always will. A digital image in the current state of things first looks like a photographic image, and most people (including trained printmakers) then see it as being a digital print.

Basically, more megapixels may be an argument to sell to the technophiles, but unless the common treatment of the photographic image changes, then I doubt new advances in technology will change the habits of the everyday shooter.

Just think about it: do you think that the difference between an 8 megapixel image and a 30 megapixel photo will make a noticable difference in a 5x7" drugstore print?

I think the point you're talking about is increasing the lattitude of the sensor, which would be an amazing and completely beneficial improvement in digital capture. A print, however, is always going to consist of shadows, midtones, and highlights. Detail in all these areas is great, but at one point a sensor with the lattitude you're anticipating is going to render every tone of everyday light as a middle value, creatting a flat, muddy image. These chips better be versatile and adjustable.

[Ian Rees]

It's a fantastic read for the lack of understanding of optical physics and CCD design that managed to get published in the New York Times, as well as blind faith in a misapplication of Moore's law. Seriosly, wtf, this guy has no idea what he's talking about.

[Ian Rees]

I should elaborate before I'm jumped on.

Moore's Law has absolutely nothing to do with CCD design; pixel densities of over 100 megapixels could easily be fabricated on an APS-C or 35mm sized sensor.

Why doesn't it exist? No one would want such a chip, because it would suck.

1) The smaller the CCD well, the worse the SNR ratio. It would have either very low sensitivity or very high noise
2) The sensor would be diffraction limited at something like f/4
3) Due to above, sensor size, and resolution, the optics would have to be awesomely precise, and very expensive

Why can't signal processing do everything? You need a lens to form an image on the detector. I don't imagine the laws of physics changing much. :)

Why aren't sensitivities going to increase massively in the future? CCDs are pretty mature; improvements are going to come from materials science, wells with better quantum efficiencies, etc. Same reason we won't see 32/64/96 bit depth sensors.

Why not just build giant sensors? Sensor price is not necessarily linked to Moore's law. Chip fabrication costs per mm^2 of die have been relatively constant for a while; there isn't the exponential like improvements seen in IC fabrication due to process shrinks and improvements. Yields decrease with larger chips. They get very expensive very fast. Probably won't see large price drops in 35mm dSLRS anytime soon.

Why not infinite DOF? Gahhhhh.... all reasons above and the dependence of DOF on focal length, which is dependent on field of view and format size.

Why not a panoptic camera, that focuses on multiple planes at the same time? One has been built, but it turns a 16 megapixel medium format back into something more like half a megapixel. Won't see this until we see CCDs with much higher (probably impossible) efficiencies.

etc. etc. etc.

[Chuck Taylor]

I don't think I could have conceived of ever reading that "this guy" Nathan Mhyrvold "has no idea what he's talking about." I love it.

So what about optics? How good are optics today and how good could they be in a consumer product -- at what point does pixel resolution become irrelevant with a really good lens?

[Ian Rees]

The wavelengths (in the range of ~500nm) of visible light set an absolute limit of resolution due to diffraction. It's a complex topic, but unless there are changes in the fundamental laws of the universe, it's a hard limit, even with theoretically perfect optics. :)

Here's a good article about it:
http://www.cambridgeincolour.com/tutorials/diffraction-photography.htm#

imho, in the real world, with current lens qualities, really more than 16 megapixels in an APS-C camera or 30 megapixels in a 35mm FF camera will be the point at which additional resolution will be very, very expensive to acquire due to demands on the lenses. We have already well exceeded this point with P&S cameras; 10 megapixels on a tiny CCD is really pushing it. Definitely driven by the marketing department and not the engineers who are smart enough to know you're not going to see that much resolution; you'll just have tiny, noisy pixels sampling above the optical resolution: meaningless measurements, offering no actual improvement in resolution.

Mhyrvold may be a very smart guy, but he wrote some very pie in the sky predictions in a field outside his expertise without understanding the hard limits that will probably never be overcome.