digital computer. refers broadly to
computational imaging techniques that enhance or extend the capabilities of
Digital Photography . The output of these techniques is an ordinary
photograph, but one that could not have been taken by a traditional camera.
The term was first used by Steve Mann , and possibly others, to describe
their own research. More recently its definition has evolved to cover a number of
subject areas in Computer Graphics , Computer Vision , and applied
Optics . These areas are given below, organized according to a taxonomy
proposed by Shree Nayar . Within each area is a list of techniques, and for
each technique one or two representative papers or books are cited.
Deliberately omitted from the
taxonomy are Image Processing (see also Digital Image Processing )
techniques applied to traditionally captured
images in order to produce better images. Examples of such techniques are
Image Scaling , dynamic range compression (i.e. Tone Mapping ),
Color Management , image completion (a.k.a. inpainting or hole filling),
Image Compression . Digital Watermarking , and artistic image effects.
Also omitted are techniques that produce Range Data ,
Volume Data , 3D Model s, 4D Light Fields ,
4D, 6D, or 8D BRDF s, or other high-dimensional image-based representations.
Controlling photographic illumination in a structured fashion, then processing the captured images,
to create new images. The applications include image-based relighting, image enhancement, geometry/material recovery and so forth.
- Flash/no-flash imaging
- --- Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K. (2004). "Digital Photography with Flash and No-flash Image pairs" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 23, No. 3, pp. 664-672.
- --- Eisemann, E., Durand, F. (2004). "Flash Photography Enhancement via Intrinsic Relighting" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 23, No. 3, pp. 673-678.
- --- Agrawal, A., Raskar, R., Nayar, S.K., Li, Y. (2005). "Removing photography artifacts using gradient projection and flash-exposure sampling" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 24, No. 3, pp. 828-835.
- Multi-flash imaging
- --- Raskar, R., Tan, K.-h., Feris, R., Yu, J., Turk, M., (2004). " Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering using Multi-Flash Imaging" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 23, No. 3, pp. 679-688.
- Light stages and domes
- --- Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M. (2000). "Acquiring the Reflectance Field of a Human Face" , ''Proc. ACM Siggraph'', ACM Press, pp. 145-156.
- --- Malzbender, T., Gelb, D., Wolters, H. (2001). "Polynomial Texture maps" , ''Proc. ACM Siggraph'', ACM Press, pp. 519-528.
- Other forms of temporally multiplexed illumination
- --- Masselus, V., Peers, P., Dutré, P., Willems, Y.D. (2003). "Relighting with 4D Incident Light Fields" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 22, No. 3, pp. 613-620
- --- Schechner, Y., Nayar., S., Belhumeur, P. (2003). "A Theory of Multiplexed Illumination" , ''Proc. ICCV 2003'', p. 808.
- Other uses of structured illumination
- --- Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., Bolas, M. (2004). "Synthetic Aperture Confocal Imaging" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 23, No. 3, pp. 825-834.
- --- Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., Lensch, H. (2005). "Dual Photography" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 24, No. 3, 2005, pp. 745-755.
- --- Nayar, S.K., Krishnan, G., Grossberg, M.D., Raskar, R. (2006). "Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 25, No. 3, pp. 935-944.
- --- Zhang, L. Nayar, S. K. (2006). "Projection Defocus Analysis for Scene Capture and Image Display" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH) Jul, (2006)
Capture of optically coded images, followed by computational decoding to produce new images.
Coded aperture imaging was mainly applied in astronomy or X-ray imaging to boost the image quality. Instead of a single pin-hole, a pinhole pattern is applied in imaging, and . The coded aperture can also improve the quality in light field acquisition using Hadamard transform optics.
- Coded aperture imaging
- --- Zand, J. (1996). "Coded Aperture Imaging in High Energy Astronomy" , NASA Laboratory for High Energy Astrophysics (LHEA) at NASA's GSFC.
- --- Levin, A. , Fergus, R., Durand, F., Freeman, B. (2007) " Image and Depth from a Conventional Camera with a Coded Aperture" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH)
- --- Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., Tumblin, J. (2007) "Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH)
- --- Liang, C.K., Liu, G., Chen, H. H. (2007). "Light Field Acquisition using Programmable Aperture Camera" , ''Proc. ICIP 2007''.
- Coded exposure imaging
- --- Raskar, .R., Agrawal, A., Tumblin, J. (2006). "Coded Exposure Photography: Motion Deblurring using Fluttered Shutter" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 25, No. 3, pp. 795-804.
- --- Agrawal, A., Raskar, .R. (2007). "Resolving Objects at Higher Resolution using a Single Motion Blurred Image" , ''IEEE Conference on Computer Vision and Pattern Recognition (CVPR)''
- Light Field Photography
- --- Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M. (2005). "High Performance Imaging Using Large Camera Arrays" , ''ACM Trans. on Graphics'' (Proc. SIGGRAPH), Vol 24, No 3, pp. 765-776.
- --- Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P. (2005). "Light Field Photography with a Hand-Held Plenoptic Camera" , Stanford Tech Report CTSR 2005-02, April, 2005.
- --- Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., Tumblin, J. (2007) "Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH)
- --- Liang, C.K., Liu, G., Chen, H. H. (2007). "Light Field Acquisition using Programmable Aperture Camera" , ''Proc. ICIP 2007''.
- Catadioptric Imaging
- --- Baker, S., Nayar, S. (1999). "A Theory of Single-Viewpoint Catadioptric Image Formation" , ''International Journal of Computer Vision'', Vol. 35, Issue 2, pp. 175-196.
- --- Kuthirummal, S., Nayar, S. K. (2006). "Multiview Radial Catadioptric Imaging for Scene Capture" ''ACM Trans. on Graphics'' (Proc. of ACM SIGGRAPH).
Processing of non-optically coded images to produce new images.
- Panorama Mosaicing
- --- Steve Mann and R. W. Picard " Virtual bellows: constructing high-quality images from video .", In Proceedings of the IEEE First International Conference on Image ProcessingAustin, Texas, November 13-16, 1994 PDF
- --- Steve Mann and R.W. Picard " Video orbits of the projective group a simple approach to featureless estimation of parameters ", IEEE Transactions on Image Processing, pp. 1281 - 1295, Vol. 6, Iss. 9, 1997
- --- Szeliski, R., Shum, H.-Y. (1997). "Creating full view panoramic image mosaics and environment maps" , ''Proc. ACM Siggraph'', ACM Press, pp. 251-258.
- Matte Extraction
- --- Joshi, N., Matusik W., Avidan S. (2006). "Natural Video Matting using Camera Arrays" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH).
- --- Chuang, Y.-Y., Curless, B., Salesin, D.H., Szeliski, R. (2001). "Video Matting of Complex Scenes" , ''ACM Transactions on Graphics'' (Proc. SIGGRAPH), Vol. 21, No. 3, pp. 243-248.
- Digital Photomontage (see also Photomontage )
- --- Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., Cohen, M. (2004). "Interactive digital photomontage" , ''Proc. ACM Siggraph'', ACM Press, pp. 294-302.
- High Dynamic Range Imaging
- --- S. Mann. Compositing multiple pictures of the same scene. In Proceedings of the 46th Annual IS&T Conference, Cambridge, Massachusetts, May 9-14 1993. The Society of Imaging Science and Technology.
- --- Mann, S. (1994). "Being 'undigital' with digital cameras: Extending dynamic range by combining differently exposed pictures" , MIT Media Laboratory Perceptual Computing Section Technical Report TR-323, also appears in ''Proc. IS&T 46th Annual Conference, May 1995, pp. 422-428.
- --- Debevec, P.E., Malik, J. (1997). "Recovering high dynamic range radiance maps from photographs" , ''Proc. ACM Siggraph'', ACM Press, pp. 369-378.
- --- Steve Mann, " Comparametric Equations with Practical Applications in Quantigraphic Image Processing ", IEEE Transactions on Image Processing, Vol 9, No.8, August, 2000
Detectors that combine sensing and
processing, typically in hardware.
Although computational photography is a currently popular buzzword, many of its
techniques first appeared in the computer vision literature,
either under other names or within papers aimed at 3D shape analysis.
A few examples are:
- Image-based relighting
- --- Hallinan, P.W. 1994. "A low-dimensional representation of human faces for arbitrary lighting conditions" , ''Proc. CVPR 1994'', pp. 995-999.
- --- Belhumeur, P.N., Kriegman, D.J. 1998. "What Is the Set of Images of an Object under All Possible Illumination Conditions?" , ''International Journal of Computer Vision (IJCV)'', Vol. 28, Issue 3, p. 245-260.
|
