User:MarcLevoy/Computational photography

Computational imaging refers to any image formation method that involves a digital computer. Computational photography is a recently invented phrase referring to computational imaging techniques that enhance or extend the capabilities of digital photography. Within this rapidly evolving area, one can identify a number of broad research directions. These are given below, with a list of techniques in each area. For each technique, one or two representative papers or books are cited.

Computational illumination
Controlling photographic illumination in a structured fashion, then processing the captured images, to create new images.


 * Flash/no-flash imaging
 * Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K. (2004). "Digital photography with flash and no-flash image pairs", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 23, No. 3, pp. 664-672.
 * Agrawal, A., Raskar, R., Nayar, S.K., Li, Y. (2005). "Removing photography artifacts using gradient projection and flash-exposure sampling", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 24, No. 3, pp. 828-835.


 * Multi-flash imaging
 * Raskar, R., Tan, K.-h., Feris, R., Yu, J., Turk, M., " Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering using Multi-Flash Imaging", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 23, No. 3, pp. 679-688.


 * Light stages and domes
 * Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M. (2000). "Acquiring the Reflectance Field of a Human Face", Proc. ACM Siggraph, ACM Press, pp. 145-156.
 * Malzbender, T., Gelb, D., Wolters, H. (2001). "Polynomial Texture maps", Proc. ACM Siggraph, ACM Press, pp. 519-528.


 * Other forms of temporally multiplexed illumination
 * Masselus, V., Peers, P., Dutré, P., Willems, Y.D. (2003). "Relighting with 4D incident light fields", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 22, No. 3, pp. 613-620
 * Schechner, Y., Nayar., S., Belhumeur, P. (2003). "A Theory of Multiplexed Illumination", Proc. ICCV 2003, p. 808.


 * Other uses of structured illumination
 * Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., Bolas, M. (2004). "Synthetic aperture confocal imaging", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 23, No. 3, pp. 825-834.
 * Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., Lensch, H. (2005). "Dual Photography", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 24, No. 3, 2005, pp. 745-755.
 * Nayar, S.K., Krishnan, G., Grossberg, M.D., Raskar, R. (2006). "Fast separation of direct and global components of a scene using high frequency illumination", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 25, No. 3, pp. 935-944.

Computational optics
Capture of optically coded images, followed by computational decoding to produce new images.


 * Coded aperture imaging
 * Zand, J. (1996). "Coded aperture imaging in high energy astronomy", NASA Laboratory for High Energy Astrophysics (LHEA) at NASA's GSFC.


 * Coded exposure imaging
 * Raskar, .R., Agrawal, A., Tumblin, J. (2006). "Coded exposure photography: motion deblurring using fluttered shutter", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 25, No. 3, pp. 795-804.


 * Light field photography
 * Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P. (2005). "Light Field Photography with a Hand-Held Plenoptic Camera", Stanford Tech Report CTSR 2005-02, April, 2005.


 * Catadioptric imaging
 * Baker, S., Nayar, S. (1999). "A Theory of Single-Viewpoint Catadioptric Image Formation", International Journal of Computer Vision, Vol. 35, Issue 2, pp. 175-196.


 * Wavefront coding
 * Cathey, W.T., Dowski, E.R. (2002). "New paradigm for imaging systems", Applied Optics, Vol. 41, No. 29, 10 October 2002.

Computational processing
Processing of non-optically coded images to produce new images.


 * Panorama mosaicing
 * Szeliski, R., Shum, H.-Y. (1997). "Creating full view panoramic image mosaics and environment maps", Proc. ACM Siggraph, ACM Press, pp. 251-258.


 * Matte extraction
 * Chuang, Y.-Y., Curless, B., Salesin, D.H., Szeliski, R. (2001). "A Bayesian Approach to Digital Matting", ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 21, No. 3, pp. 243-248.


 * Digital photomontage (see also photomontage)
 * Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., Cohen, M. (2004). "Interactive digital photomontage", Proc. ACM Siggraph, ACM Press, pp. 294-302.


 * Super-resolution
 * Baker, S., Kanade, T. (2002). "Limits on super-resolution and how to break them", IEEE Trans. PAMI, Vol. 24, No. 8, August, 2002.


 * High dynamic range imaging
 * Mann, S. (1993). "Compositing Multiple Pictures of the Same Scene", The Society of Imaging Science and Technology, May 9-14, 1993, pages 50-52.
 * Debevec, P.E., Malik, J. (1997). "Recovering high dynamic range radiance maps from photographs", Proc. ACM Siggraph, ACM Press, pp. 369-378.

Computational sensors
Detectors that combine sensing and processing, typically in hardware.


 * Artificial retinas
 * Retinex sensors
 * High dynamic range sensors

Note: The foregoing taxonomy was originally proposed by Shree Nayar. Deliberately omitted from the taxonomy are image processing (see also digital image processing) techniques that are applied to conventionally captured images in order to produce better images. Examples of such techniques are image scaling, dynamic range compression (i.e. tone mapping), color management, image completion (a.k.a. inpainting or hole filling), image compression. digital watermarking, and artistic image effects. Also omitted are techniques that produce range data, volume data, 3D models, 4D light fields, 4D, 6D, or 8D BRDFs, or other high-dimensional image-based representations.

Overviews

 * Special issue on Computational Photography, IEEE Computer, August 2005.
 * Raskar, R., Tumblin, J., Computational Photography, A.K. Peters. In press.

Symposia

 * Symposium on Computational Photography and Video (MIT, May 23-25, 2005)

Courses

 * Stanford's CS 448A (Marc Levoy)
 * Northeastern University's CSG242 (Ramesh Raskar)
 * Georgia Tech's CS 4803CP (Irfan Essa)
 * MIT's 6.098/6.882 (Fredo Durand and Bill Freeman)
 * SIGGRAPH 2006 course on Computational Photography (Ramesh Raskar and Jack Tumblin)