EE367 / CS448I: Computational Imaging

Course Description

Computational imaging systems have a wide range of applications in consumer electronics, scientific imaging, HCI, medical imaging, microscopy, and remote sensing. We discuss digital photography and basic image processing, convolutional neural networks for image processing, denoising, deconvolution, single pixel imaging, inverse problems in imaging, proximal gradient methods, introduction to wave optics, time-of-flight imaging, end-to-end optimization of optics and imaging processing. Emphasis is on applied image processing and solving inverse problems using classic algorithms, formal optimization, and modern artificial intelligence techniques. Students learn to apply material by implementing and investigating image processing algorithms in Python. Term project. Recommended: EE261, EE263, EE278. (Course Catalog Entry)

Topics include:

Human visual perception
Digital cameras and ISPs
Denoising, deconvolution, and other inverse problems in imaging
Convolutional neural networks for solving inverse problems
Proximal gradient methods / formal optimization for solving inverse problems
High dynamic range imaging
Light field imaging
Introduction to wave optics
End-to-end optimization of optics and image processing
... more interesting topics.

Course Goals

Students will learn about computational imaging methods and applications with a focus on solving inverse problems in imaging, such as denoising, deconvolution, single-pixel imaging, and others. For this purpose, we will discuss classic algorithms, modern data-driven approaches using convolutional neural networks (CNNs), and also proximal gradient methods, including half-quadratic splitting and ADMM, that combine formal optimization with CNNs. The homeworks require programming and image processing in Python.

Instructors

Gordon Wetzstein

Teaching Assistants

Qingqing Zhao

Axel Levy

Class Time and Lecture Format

Class is on Mondays and Wednesdays 1:30-2:50pm in Y2E2 111. All lectures are recorded and the videos will be available on canvas in a timely manner.

Problem Sessions

Fridays 2:30-3:30pm (TA), Lathrop 294, starting on January 12 for the first 6 weeks of the quarter

Office Hours

Mondays 3-4pm (Gordon), Packard 236 & zoom (hybrid), starting on January 8; discussion about course material and projects; zoom link on canvas

Wednesdays 3-4:30pm (TA), Packard 104 & Zoom (hybrid), starting on January 10; questions about homework, labs and lectures; zoom link on canvas

Contact

Email (ee367-win2324-staff@lists.stanford.edu) us ONLY when your problems cannot be resolved via Ed Discussion.

Schedule and Syllabus

The lecture videos are released weekly on canvas.

Students with Documented Disabilities: Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Office of Accessible Education (OAE). Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty. Unless the student has a temporary disability, Accommodation letters are issued for the entire academic year. Students should contact the OAE as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 723-1066, URL: https://oae.stanford.edu/).

This is the syllabus for the Winter 2024 iteration of the course.

Week	Date	Event	Description	Material	Readings
Week 1	Mon 1/8	Lecture 1	Introduction and fast forward Overview of class, logistics, discussion of project ideas	[slides]
	Wed 1/10	Lecture 2	The human visual system Perception of color, depth, contrast, resolution, ...	[slides]	Hybrid Images Paper
	Wed 1/10		HW1 out	[link]
	Fri 1/12		Problem session 1	[slides]
Week 2	Mon 1/15		MLK Day (No Lecture)
	Wed 1/17	Lecture 3	Digital photography I Ray optics, aperture, depth of field, exposure, sensor, noise	[slides]	Archived Course CS178
	Wed 1/17		HW2 out	[link]
	Fri 1/19		Problem session 2	[slides]
	Fri 1/19		Homework #1 due at 11:59pm
Week 3	Mon 1/22	Lecture 4	Digital photography II CameraISP, demosaicking, denoising, deconvolution	[slides]	Demosaicking Paper Non-local Means Paper Intro to Bilateral Filtering
	Wed 1/24	Lecture 5	Math review Quick review of sampling, optimization, deconvolution, ...	[slides]
	Wed 1/24		HW3 out	[link]
	Fri 1/26		Problem session 3	[slides]
	Fri 1/26		Homework #2 due at 11:59pm
Week 4	Mon 1/29	Lecture 6	Great ideas in computational photography HDR, tone mapping, coded apertures, flutter shutter	[slides]	HDR Imaging Paper Tone Mapping Paper Ext. Depth of Field Paper Flutter Shutter Paper Learned Coded Apertures Neural Sensors
	Wed 1/31	Lecture 7	Guest Lecture by Dr. Orly Liba Computational photography at Google (burst imaging, night sight, ...)	[slides]
	Wed 1/31		HW4 out	[link]
	Fri 2/2		Problem session 4	[slides]
	Fri 2/2		Homework #3 due at 11:59pm
Week 5	Mon 2/5	Lecture 8	Introduction to neural networks MLPs, CNNs, ResNets, denoising with CNNs	[slides]
	Wed 2/7	Lecture 9	Solving inverse problems with neural networks UNet, deconvolution with CNNs, ...	[slides]
	Wed 2/7		HW5 out	[link]
	Fri 2/9		Problem session 5	slides
	Fri 2/9		Homework #4 due at 11:59pm
Week 6	Mon 2/12	Lecture 10	Image deconvolution with HQS natural image priors, half quadratic splitting (HQS), and efficient deconvolution with image priors	[slides]	Lecture 10 Notes
	Wed 2/14	Lecture 11	Solving regularized inverse problems with ADMM Single-pixel imaging, ADMM, solving general inverse problems	[slides]	Lecture 11 Notes
	Wed 2/14		HW6 out	[link]
	Fri 2/16		Problem session 6	[link]
	Fri 2/16		Homework #5 due at 11:59pm
Week 7	Mon 2/19		President's Day (No Lecture)
	Wed 2/21	Lecture 12	Introduction to wave optics and deep optics free-space wave propagation, diffraction limit, end-to-end optimization of optics and image processing using AI	[slides]	Deep Optics AI using Optics
	Fri 2/23		Project Proposal due at 11:59pm
	Fri 2/23		Homework #6 due at 11:59pm
Week 8	Mon 2/26	Lecture 13	Phase retrieval and holography phase retrieval, Fourier ptychography, computer-generated holography	[slides]
	Wed 2/28		Midterm
Week 9	Mon 3/4	Lecture 14	Guest Lecture by Dr. Katie Bouman on zoom Imaging black holes	[slides]
	Wed 3/6	Lecture 15	Guest Lecture by Dr. David Lindell on zoom Time-of-flight, single-photon, and non-line-of-sight imaging	[slides]
Week 10	Mon 3/11	Lecture 16	Guest Lecture by Axel Levy Computational imaging for structural biology: cryo-EM, x-ray, and more	[slides]
	Wed 3/13		Final Project - poster presentation 1:30-3:00pm, Packard Atrium, 36(W)*24(H)		ProjectPosterTemplate.ppt
	Fri 3/15		Final Project - report and code due on 3/15/2024, 11:59pm		iccp2020_latex_template.zip

Textbooks, Course Notes, and other Reading Materials

There is no textbook for this course but students are required to read additional resources as indicated for each lecture or week. These readings will help get a better intuition and deeper insights into the topics of this course. The readings may also be required to complete tasks for assignments.

Detailed course notes are available for the topics covered in week 6: deconvolution, single-pixel imaging, and solving regularized inverse problems; the half-quadratic splitting (HQS) method; the alternating direction of multipliers (ADMM); noise and image reconstruction. orientation tracking with inertial measurement units (IMUs) and pose tracking with the VRduino. You can download these notes here:

Asking Questions and Getting Help

If you have a question, to get a response from the teaching staff quickly we strongly encourage you to post it to the class Ed Discussion on canvas. For private matters, please make a private note visible only to the course instructors. For longer discussions with TAs and to get help in person, we strongly encourage you to come to office hours. You can also reach out to us via email using the course staff mailing list.

Gradescope

All assignments, your midterm, and the final project videos and reports should be submitted on Gradescope. Use the code VB64PD to join the class. If you work as a team, make sure to indicate your team member in the submission

Assignment Details

There will be weekly assignments (see syllabus) in this class. These assignments will contain some theoretical questions and also implementations of techniques that we will discuss in class. Please refer to assignment writeups for details. After you finish, submit your code and report on Gradescope.

Grading Policy

The course requirements include: (1) 6 assignments, (2) an in-class or virtual midterm (80 mins long), and (3) a major final project, including a project proposal, final ~6 page conference paper-style report, source code, and a poster (or video) presentation.

Assignments: 50%
Midterm: 20%
Final Project: 30%

There are no "late days" for the assignments. If you choose to submit an assignment late, we will accept it for up to 24h after the submission deadline with a 30% penalty (final grade multiplied by 0.7).

The final project grade takes into account your poster presentation (organization of poster, clarify of presentation, ability to answer question), your source code submission (code organization and documentation), and your final project report (approriate format and length, abstract, introduction, related work, description of your method, quantitative and qualitative evaluation of your method, results, discussion & conclusion, bibliography).

Course Projects, Project Proposal, and Final Report

You can work in teams of up to 3 students for the project. Submit only one proposal and final report for each team. The expected amount of work is relative to the number of team members, so if two teams work on a similar project, we'd expect less work from a smaller team. Before you start to work on the proposal or the report, take a look at some of the past project proposals and reports to give you sense for what's expected.

The project proposal is a 1-2 page document that should contain the following elements: clear motivation of your idea, a discussion of related work along at least 3 scientific references (i.e., scientific papers not blog articles or websites), an overview of what exactly your project is about and what the final goals are, milestones for your team with a timeline and intermediate goals. Once you send us your proposal, we may ask you to revise it and we will assign a project mentor to your team.

The final project report should look like a short (~6 pages) conference paper. We expect the following sections, which are standard practice for conference papers: abstract, introduction, related work, theory (i.e., your approach), analysis and evaluation, results, discussion and conclusion, references. To make your life easier, we provide an LaTex template that you can use to get started on your report (see syllabus for link).

EE367 / CS448I: Computational Imaging

Winter 2024