CS 8803 PHO (3-0-3): Advanced Computational Photography – (Instructor: Irfan Essa, Grant Schindler) – This class explores perceptual and technical aspects of pictures, and more precisely the capture and depiction of reality on a 2D medium. The scientific, perceptual, and artistic principles behind image-making will be emphasized. Topics include the relationship between pictorial techniques and the human visual system; intrinsic limitations of 2D representations and their possible compensations; and technical issues involving depiction. Technical aspects of image capture and rendering, and exploration of how such a medium can be used to its maximum potential, will be examined. Students are strongly encouraged (not required) to bring their digital cameras and a laptop to facilitate experiments.
For more information, see the slides (link).
- Professor Irfan Essa (Email is the BEST (and ONLY) option: irfan at cc dot gatech dot edu, please use CS8803PHO: as the first words in the subject line)
- Dr. Grant Schindler (Email is the BEST (and ONLY) option: schindler at cc dot gatech dot edu, please use CS8803PHO: as the first words in the subject line)
- Office Hours: After class OR schedule via email appointment
- Location: KACB 2456
- Monday – Wednesday 4:35p – 5:55p.
In this class. you will learn about:
- The Medium of Photography (History to Modern Technologies)
- Digital Imaging (Sensors to Software)
- Computational Processes associated with Imaging and Photography
- Computation as applied to Digital Media.
- Image Analysis and Synthesis
- Images and Photography on the Web/Internet
- Storytelling with images (and computation).
Text & Reading Material.
- [RS] Computer Vision: Algorithms and Applications; by Richard Szeliski (see link for a draft of this book available online)
Other material available online or on T-square (See Resources/Readings/ on T-Square site for this class).
Assignments and Grading
- Class Attendance & Participation (15 %)
- Assignments / Homeworks (20 %) [There will be 2-3 Assignments]
- In Class Presentation (15%)
- Reviews of Assigned Readings (15%)
- Final Project (35%)
- Includes: Proposal/Teaming(5%), Updates (7%), In class presentation and demo (15%), Final Report and Self Evaluation (8%).
- All of the above subject to slight modifications as needed, which will be announced in class.
- Class attendance is required. Late by 15 minutes, counts as an absence. Legitimate reasons for being excused from class include, personal issues, health (keep those germs away from class), interview, conference travel, etc. Travelling and exploring, assignments due in other classes, out to pick up friends, and other such excuses not accepted. Please inform Instructor of a planned absence via email before class.
- Grading. Usually a score of 90 is considered an “A” and 100 an “A+” (but a 100 is given to only artifacts that are exceptional and beyond what was expected!). 80, 70 are “B” and “C” respectively.
- Homeworks Assignments will be graded on a list of criteria (specified on the assignment) such as quality of work, completeness, insight into technical issues, insight into other relevant issues, etc. Assignments are due at the start of class on the day they are due.
- Late Assignments: Everything is DUE before the class session. NO extensions. In most circumstance, students will be asked to discuss their assignment in class.
- Laptops use in class: Use of laptops in class room for purposes of note-taking and work related to the class is allowed, but ONLY and ONLY for that purpose. If a student is seen surfing the web during class, or chatting with someone, or emailing, then points will be deducted from the class attendance and participation portions of the grade (3% for each infraction, with a total of 3 max, after that the student looses the whole class participation score).
- Cellphones in class: Please turn your cellphone and other mobile devices to “silent” mode during class. Thanks.
- This class abides by the Georgia Tech Honor Code. All assigned work is expected to be individual, except where explicitly written otherwise. You are encouraged to discuss the assignments with your classmates; however, what you hand in should be your own work. If any work product was produced based on discussions with someone else (in the class OR outside), please specify clearly in the final turn-in.
Assignments and ideas on this syllabus build on those from everyone who has taught this material before.
Here is a weekly and day by day schedule of the class. See T-Square @ GATech site for this class for details on readings and assignments. Some topics and readings are subject to change, so please make sure to check this site on weekly basis (at-least). Links are provided to slides and material in class to the GT Internal T-Square site.
Week 1 (1/9/12)
- TOPIC: Introduction and Overview [IE/GS]
- MATERIAL from lecture: See T-square Site (01_Introduction)
- Topic: CAMERAS [IE] (How do cameras work? From a pinhole camera to a digital camera.)
- MATERIAL from lecture: See T-square Site (02_Cameras)
Week 2 (1/16/12)
- MLK HOLIDAY – No Class
- TOPIC: PHOTOGRAPHY [IE] (What is the medium of photography? What is it’s technology?)
- MATERIAL from lecture: See T-square Site (03_Photography)
Week 3 (1/23/12)
- TODO: Assignment #0 Presentations in class
- TODO: Assignment #1 Out
Week 4 (1/30/12)
- TOPIC: Photo Synth/Photo Tourism [IE]
- Noah Snavely, Steven M. Seitz, Richard Szeliski, “Photo tourism: Exploring photo collections in 3D,” ACM Transactions on Graphics (SIGGRAPH Proceedings), 25(3), 2006, 835-846. [pdf] [bibtex].
- Noah Snavely, Steven M. Seitz, Richard Szeliski, “Modeling the world from Internet photo collections,” International Journal of Computer Vision (to be published). [pdf] [bibtex]
- TODO: Assignment #1 Out
- MORE INFO
- TODO: Assignment #2 Out
- Wednesday (Irfan Away)
- TOPIC: Introduction to Structure from Motion I [GS]
- READINGS: Grant Schindler, Frank Dellaert. 4D Cities: Analyzing, Visualizing, and Interacting with Historical Urban Photo Collections. (To be published) [pdf: T-Square]
Week 5 (2/6/12)
- TOPIC: Introduction to Structure from Motion II [GS]
- Paul E. Debevec, Camillo J. Taylor, Jitendra Malik (1996) “Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach” SIGGRAPH 1996. (PDF)
- M. Brown and D. G. Lowe. “Recognizing Panoramas.” ICCV 2003. (PDF)
- Yasutaka Furukawa, Brian Curless, Steven M. Seitz, Richard Szeliski “Reconstructing Building Interiors from Images” ICCV 2009. (PDF)
- TOPIC: Introduction to Structure from Motion III [GS, IE]
- Image-based interactive exploration of real-world environments.
Matthew Uyttendaele, Antonio Criminisi, Sing Bing Kang, Simon Winder, Richard Szeliski, Richard Hartley. Computer Graphics and Applications 2004.
- Photographing long scenes with multi-viewpoint panoramas.
Aseem Agarwala, Maneesh Agrawala, Michael Cohen, David Salesin, Rick Szeliski. SIGGRAPH 2006.
Related: http://research.microsoft.com/en-us/um/people/kopf/street_slide/index.html (Watch video!)
- Building Rome in a Day.
Sameer Agarwal, Noah Snavely, Ian Simon, Steven M. Seitz, Richard Szeliski. ICCV 2009.
Related: http://grail.cs.washington.edu/software/cmvs/ (Watch video!)
- Image-based interactive exploration of real-world environments.
Week 6 (2/13/12)
- TOPIC: FCam [IE]
- Andrew Adams, Eino-Ville (Eddy) Talvala, Sung Hee Park, David E. Jacobs, Boris Ajdin, Natasha Gelfand, Jennifer Dolson, Daniel Vaquero, Jongmin Baek, Marius Tico, Henrik P.A. Lensch, Wojciech Matusik, Kari Pulli, Mark Horowitz, Marc Levoy (2010), “The Frankencamera: An Experimental Platform for Computational Photography Proc. SIGGRAPH 2010
- Andrew Adams Natasha Gelfand, Jennifer Dolson, Marc Levoy (2009) “Gaussian KD-Trees for Fast High-Dimensional Filtering” Proc. SIGGRAPH 2009
- Marc Levoy (2010) “Experimental Platforms for Computational Photography “IEEE Computer Graphics and Applications, Vol. 30, No. 5, September/October, 2010, pp. 81-87.
- MORE INFO:
- TODO: ASSIGNMENT #3 OUT
- TOPIC: Image/Texture Syntesis & Blending [IE]
- Burt and Adelson (1983) “The Laplacian Pyramid as a Compact Image Code”, In IEEE Transactions on Communications, 31 (4). p 532-540. 1983 (PDF) (DOI)
- Burt and Adelson (1983) “A multiresolution spline with application to image mosaics”. In ACM Transactions on Graphics, 2 (4). 1983 (PDF) (DOI)
- Texture Synthesis
- Heeger & Bergen (1995), “Pyramid-based texture analysis/synthesis”, IN ACM SIGGRAPH 1995 (PDF) (DOI)
- Efros & Lueng (1999), “Texture Synthesis by Non-parametric Sampling” ICCV 1999 (Website)[PDF]
- DeBonet (1997), “Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images”, SIGGRAPH 1997. (Website) [PDF]
Week 7 (2/20/12)
- TOPIC: Large Image Databases (GS)
- James Hays, Alexei A. Efros. Scene Completion Using Millions of Photographs Siggraph 2007. http://graphics.cs.cmu.edu/projects/scene-completion/scene-completion.pdf, http://graphics.cs.cmu.edu/projects/scene-completion/
- A. Torralba, R. Fergus, W. T. Freeman. 80 Million Tiny Images, PAMI 2008. http://people.csail.mit.edu/torralba/tmp/tiny.pdf, http://groups.csail.mit.edu/vision/TinyImages/
- David Crandall, Lars Backstrom, Daniel Huttenlocher and Jon Kleinberg, Mapping the World’s Photos, WWW 2009. http://www.cs.cornell.edu/~dph/papers/photomap-www09.pdf
- TOPIC: Texture Synthesis II and Image Stiching [IE]
Week 8 (2/27/12)
- TOPIC: Image Analogies & Synthesis (IE)
- DISCUSSION OF FINAL PROJECTS
Week 9 (3/5/12)
Week 10 (3/12/12)
- TOPIC: Plenoptic Imaging (IE)
- Adelson and Bergen (1991), “The Plenoptic Function and the Elements of Early Vision” Computational models of visual processing. [PDF][DOI]
- Adelson and Wang (1992) “Single lens stereo with a plenoptic camera”, IEEE PAMI 14(2) [PDF][DOI]
- Ng, Levoy, et al. (2005), “Light field photography with a hand-held plenoptic camera” Stanford Tech Report CTSR 2005-02, 2005. [PDF][DOI]
Week 11 (3/19/12) [SPRING BREAK]
Week 12 (3/26/12)
- TOPIC: Light Fields (IE)
- McMillan and Bishop (1995), “Plenoptic modeling: An image-based rendering system”, SIGGRAPH 1995 [PDF][DOI]
- GORTLER, S. J., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. F. 1996. The lumigraph. In SIGGRAPH 96, 43–54. [PDF][DOI]
- LEVOY, M., AND HANRAHAN, P. 1996. Light field rendering. In SIG- GRAPH 96, 31–42.[PDF][DOI]
- TOPIC: Video (IE)
- Schödl, Szeliski, Salesin, Essa (2000), “Video textures,” in ACM SIGGRAPH 2000 [PDF][DOI]
- Kwatra, Schödl, Essa, Turk, Bobick (2003), “Graphcut textures: image and video synthesis using graph cuts” SIGGRAPH 2003 [PDF] [DOI].
- Agarwala, Zheng, Pal, Agrawala, Cohen, Curless, Salesin, and Szeliski (2005) “Panoramic video textures.” SIGGRAPH 2005 [PDF][DOI]
Week 13 (4/2/12)
- TOPIC: TOPIC: Video Segmentation / Video Retargeting [MG]
- M. Grundmann, V. Kwatra, M. Han, and I. Essa (2010), “Efficient Hierarchical Graph-Based Video Segmentation,” in CVPR, 2010. [PDF][DOI]
- M. Grundmann, V. Kwatra, M. Han, and I. Essa (2010), “Discontinuous Seam-Carving for Video Retargeting,” in CVPR, 2010. [PDF][DOI]
- Avidan, Shamir (2007), “Seam Carving for Content-Based Image Retargeting”, in SIGGRAPH 2007. [PDF]
Week 14 (4/9/12)
- TOPIC: Video and Image Stabilization [MG]
- M. Grundmann, V. Kwatra, and I. Essa (2011), “Auto-Directed Video Stabilization with Robust L1 Optimal Camera Paths,” in CVPR, 2011. [PDF]
- M. Grundmann, V. Kwatra, and I. Essa (2012), “Calibration-Free Rolling Shutter Removal”, International Conference on Computational Photography, 2012 (to appear) [pdf]
- TOPIC: In Class Project Updates (Submit an update via T-square).
Week 15 (4/16/12)
Week 16 (4/23/11)
- TOPIC: Time-Lapse
- PROJECT PRESENTATIONS