EE367/CS448I: Computational Imaging and Display
Lecture: Mondays and Wednesdays, 4:30-5:50 pm, Skilling Auditorium
Problem session: Fridays, 3:30-4:20 pm, Gates B01
Instructors: Gordon Wetzstein, Orly Liba (TA), Timon Ruban (TA)
Course DescriptionComputational imaging systems have a wide range of applications in consumer electronics, scientific imaging, HCI, medical imaging, microscopy, and remote sensing. We discuss light fields, time-of-flight cameras, multispectral imaging, thermal IR, computational microscopy, compressive imaging, computed tomography, computational light transport, compressive displays, phase space, and other topics at the convergence of applied mathematics, optics, and high-performance computing related to imaging. Hands-on assignments. Prerequisites: EE 261 or equivalent (basic signal processing) and EE 263 or equivalent (linear systems/algebra). Course Catalog Entry
- Cameras and image formation
- Human visual perception
- Burst photography
- Computational microscopy
- Computational display
- Google glass, Oculus Rift, HMDs
- Structured illumination
- MS Kinect, time-of-flight cameras
- High dynamic range
- Depth and defocus
- Flash / no flash photography
- Coded aperture photography
- ... more interesting topics.
Helpful BackgroundThis course requires programming experience (especially Matlab) as well as knowledge of linear algebra, basic calculus, and optimization. The second class will review most of the required mathematical concepts (see tentative schedule below). Previous knowledge of computer graphics and computer vision will be helpful.
Courses that will be very helpful, but which are not absolutely required:
- EE 263, Linear Dynamical Systems (or equivalent, such as CS 205A)
- EE 278, Introduction to Statistical Signal Processing (or equivalent)
- EE 261, The Fourier Transform and its Applications (or equivalent)
- CS 148, Introduction to Computer Graphics and Imaging
- PSYCH 221, Applied Vision and Image Systems Engineering
- EE 364A, Convex Optimization I
- EE 368, Digital Image Processing
Important Class Information
- Contact: firstname.lastname@example.org
- Lectures: Mondays and Wednesdays, 4:30-5:50 pm, Skilling Auditorium
- TA problem sessions: Fridays, 3:30-4:20 pm, Gates B01
- TA office hour: Wednesdays before class (3:20-4:20 pm), Room Packard 214
- Instructor office hour: Mondays (and on Wed. 1/25) before class (3:20-4:20 pm), Packard 236
- Piazza (code EE367W2017): https://piazza.com/class/isgd64rpgps6el
- Homework submissions: https://gradescope.com
- SCPD website
- Teaching lab: Packard room 021, PCs with Matlab installed, TA will provide access (email us for remote access)
Requirements and Grading
The course requirements include: (1) six assignments, (2) an in-class midterm (80 minutes long), and (3) a major final project, including a project proposal, final report, source code, and a poster (or video) presentation.Your final grade will be made up from
- 6 assignments: 40% total
- midterm: 20%
- major final project: 40%
EquipmentDifferent equipment will be available for use in the projects: Intel RealSense, Lytro Illum, Time of Flight cameras, machine vision and SLR cameras, The Eye Tribe (gaze tracker), Olympus Air / Open Camera Platform, ...
TextbookNo textbook, students are expected to read relevant literature as discussed in class.
Pinhole Camera GalleryYou can find some notable pinhole camera photos from previous offerings here
|Class Date||Topic||Details||Slides||Additional Readings||Assignments|
|Introduction and fast forward||overview of class, logistics, discussion of project ideas||lecture1.pdf|
|The human visual system||perception of color, depth, contrast, resolution||lecture2.pdf||- Hybrid images paper||HW1|
|Martin Luther King Day - no class!|
|Digital photography I||optics, aperture, depth of field, exposure, noise, sensors||lecture3.pdf||- archived course CS 178||HW2|
|Problem session||PS2||HW1 due at noon|
|Digital photography II||image processing pipeline||lecture4.pdf||- Demosaicing paper
- Non-local means paper
- Intro to bilateral filter
|Sampling, Linear Systems||review of sampling, regularized linear systems||lecture5.pdf||HW3||
|Problem session||PS3||HW2 due at noon|
|Deconvolution||inverse filtering, Wiener filtering, total variation, ADMM||lecture6.pdf||- Lecture notes: deconvolution
- ADMM paper
|Burst photography||HDR, tone mapping, super-resolution, flash/no-flash, multi-flash||lecture7.pdf||- HDR paper
- Tone mapping paper
|Problem session||PS4||HW3 due at noon|
|Light field photography||camera arrays, lytro, coded masks, refocus, fourier slice theorem||lecture8.pdf||- original light field paper
- other light field paper
- light field thesis
|Coded computational photography||extended depth of field, motion invariance, flutter shutter||lecture9.pdf||- flutter shutter paper||HW5|
|Problem session||PS5||HW4 due at noon|
|Noise||lecture10.pdf||- Lecture notes: noise and deconvolution with noise||Project proposal due|
|Compressive imaging||single pixel camera, compressive sensing, compressive hyperspectral imaging, compressive light field imaging||lecture11.pdf||- Lecture notes: single pixel camera||HW6|
|Problem session||PS6||HW5 due at noon|
|President's day - no class!|
|Computational illumination and light transport||Structured illumination, photometric stereo, shape from secularities, optical computing||lecture12.pdf|
|HW6 due at noon|
|Displays blocks||LCDs, SLMs, OLEDs, stereo displays, light field displays||lecture14.pdf|
|Computational displays||HDR displays, projection displays, vision-correcting displays, volumetric displays||lecture15.pdf|
|Wearable displays||head-mounted displays (HMDs), virtual reality (VR), augmented reality (AR)||lecture16.pdf|
|Final project poster presentation
Packard Atrium, 4:30-6:30pm
Poster printing instructions
|Project reports and code due (until Fri, 3/17, midnight)|
|Tue, Mar 28||Grades due|
Computing ResourcesThe computers in the Stanford Center for Image Systems Engineering (SCIEN) Lab can be used to do your work in this class, although you can choose to use other university machines or your own computer. These machines are located in Room 021 (basement) of the Packard building. Room 021 is protected by a door key, which can be obtained by emailing the course staff.
The SCIEN computers are equipped with MATLAB (with the Image Processing Toolbox). They use the same username/password login as your normal Leland account, and all your regular files on the Leland network appear when you log into a SCIEN computer. The SCIEN machines can be remotely accessed using Microsoft Remote Desktop on the Stanford network. The device names are rm021-1.stanford.edu through rm021-20.stanford.edu. The login is win\SUNetID and the password is your Leland password. If connecting from off campus, you will need to use the Stanford VPN service. Only two students can simultaneously access one given device, so you may want to try another one if your access gets denied. If you have difficulty accessing the lab resources, please reach out to Steven Clark (email@example.com) or John DeSilva (firstname.lastname@example.org) for help. For basic tutorials on MATLAB, please look here.
Poster Printing InstructionsTo have your poster printed, send your poster in .pdf format to email@example.com.
Send the poster at least 3 hours before you want it printed (for our poster session this is 1:30 pm on 3/15).
Please include the dimensions (size) of your poster. See the poster template for the suggested size and design.
Please include the class / conference / event where the poster will be presented.
When your poster is ready you will receive an email telling you where your poster may be picked up.
For more info: Poster printing instructions