Computational Photography Course ID 15663 Description Computational photography is the convergence of computer graphics, computer vision and imaging. Its role is to overcome the limitations of the traditional camera, by combining imaging and computation to enable new and enhanced ways of capturing, representing, and interacting with the physical world. This advanced undergraduate course provides a comprehensive overview of the state of the art in computational photography. At the start of the course, we will study modern image processing pipelines, including those encountered on mobile phone and DSLR cameras, and advanced image and video editing algorithms. Then we will proceed to learn about the physical and computational aspects of tasks such as 3D scanning, coded photography, lightfield imaging, time-of-flight imaging, VR/AR displays, and computational light transport. Near the end of the course, we will discuss active research topics, such as creating cameras that capture video at the speed of light, cameras that look around walls, or cameras that can see through tissue. The course has a strong hands-on component, in the form of seven homework assignments and a final project. In the homework assignments, students will have the opportunity to implement many of the techniques covered in the class, by both acquiring their own images of indoor and outdoor scenes and developing the computational tools needed to extract information from them. For their final projects, students will have the choice to use modern sensors provided by the instructors (lightfield cameras, time-of-flight cameras, depth sensors, structured light systems, etc.). Key Topics Digital Photography, Pinholes and lenses, Photographic optics and aberrations, Exposrue and high-dynamic-range imaging, Noise, Color, Bilateral and edge-aware filtering, Gradient-domain image processing, Focal stacks and depth from (de)focus, Lightfields, Deconvolution, Coded photography, Radiometry and reflectance, Photometric stereo, Geometric camera modeling and calibration, Two-view geometry, Disparity and stereo, Time-of-flight imaging, Global illumination, Light transport matrices, Light transport probing, Fourier optics Required Background Knowledge This course requires familiarity with linear algebra, calculus, programming, and doing computations with images. If you want to enroll but have not satisfied the prerequisite courses, please make sure to contact the instructor! We make a lot of exceptions each year, on a case-by-case basis. The course does not require prior experience with photography or imaging. Course Relevance Computational photography is currently one of the most active research and industry areas within computer vision and computer graphics, with applications in a broad range of areas, including consumer and mobile photography, AR/VR/XR, medical imaging, industrial fabrication, remote sensing, autonomous driving, and robotics. 15-463 is for undergraduates; master students should register for 15-663; PhD students should register for 15-862. Course Goals The course aims to provide students with the theoretical, programming, and hardware skills necessary for understanding and implementing state-of-the-art computational photography techniques, including 3D sensing, high-dynamic-range and digital photography, lightfield imaging, time-of-flight imaging, and computational light transport. Learning Resources Additionally readings may be assigned from the following textbooks, which can also be useful references in general. All of them are available online from the CMU library: Computer Vision: Algorithms and Applications, by Richard Szeliski. Multiple View Geometry in Computer Vision, by Richard Hartley and Andrew Zisserman. Computer Vision: A Modern Approach, by David Forsyth and Jean Ponce. Foundations of 3D Computer Graphics, by Steven Gortler. Digital Image Processing, by Rafael Gonzalez and Richard Woods. Photography, by Barbara London and John Upton. Assessment Structure The final grade will be made up of: Six two-week homework assignments (60%). Final project (35%). Class participation (5%) All homework assignments will have a programming component and a photography component, where students will use a DSLR camera to capture and process their own images. The programming component of all assignments will be done in Python. At around the halfway point of the semester, students will propose a final project, to be completed by the end of the semester. Final project presentations will take place during the final exam week, and final project reports will be due before the final grade deadline. Details are available in the final project page in the syllabus link below. Extra Time Commitment The course has an optional biweekly reading group, where we go over research papers in detail. Course Link http://graphics.cs.cmu.edu/courses/15-463/
