Categories
OVERVIEW
This course is designed for advanced undergraduates, and graduate students. The course covers the latest innovations in neural rendering that have developed in the field of computer graphics over the last year. Programming assignments will help students develop GPU programming skills while implementing concepts learned in lectures and readings using deep learning APIs on a GPU cluster.
OBJECTIVES
This course introduces students to techniques that use neural networks to generate photorealistic scenes and animation in computer graphics. Course materials combine machine learning with fundamental principles from computer graphics to control scene properties including illumination, camera parameters, geometry, appearance, pose, and semantic structure. We first cover the fundamentals of computer graphics and deep learning that are relevant to neural rendering. Next, we study neural rendering methods for relighting, light transfer denoising, novel view synthesis, animation, volumetric rendering with neural radiance fields (NeRFs), Gaussian splatting, and photo-realistic avatar creation for virtual and augmented reality. Students learn the techniques by implementing a series of interactive computer programs using deep learning APIs on a GPU cluster, discussing the latest innovations (from SIGGRAPH and related venues) and by proposing and implementing a final project.
STUDENT LEARNING OUTCOMES
1. Ability to identify, formulate, and solve complex engineering problems by applying principles of engineering, science, and mathematics.
2. Ability to develop and conduct appropriate experimentation, analyze and interpret data, and use engineering judgment to draw conclusions.
3. Ability to communicate effectively with a range of audiences.
4. Ability to function effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives.
5. Ability to recognize ethical and professional responsibilities in engineering situations and make informed judgments, which consider the impact of solutions in global, economic, environmental, and societal contexts.
Time: MW 2:40pm-3:55pm
Instructor: Dr. Corey Toler-Franklin, Computer Science Department, Barnard College, Columbia University
Contact: Office Milstein 502B, ct3219@columbia.edu
Office Hours: MW 4:10pm – 5:10pm
Location: Milstein 502B
Teaching Assistant: TBD
Office Hours: TBD
Course Management: CourseWorks
PREREQUISITES:
Programming proficiency in Python and/or C/C++ (COMS W3157 Advanced Programming or similar), Linear Algebra (vector, matrix), Multivariable calculus (partial derivative, gradient, Jacobian).
In addition, we expect students to be familiar with the basics of machine learning and working with neural networks (COMS W4701 strongly recommended). We recommend some introductory knowledge of computer graphics (COMS W4160 or similar).
**Contact instructor if you are not sure you are prepared for the course**
Required Course Materials and Required Readings:
Instructor provided:
Course Notes – Volumetric Rendering & Nerfs
Course Notes – Gaussian Splatting
Selected SIGGRAPH Papers
OPTIONAL TEXTBOOK
  |
Title: Understanding Deep Learning
Author: Simon J.D. Prince |
SYLLABUS
CCOMS W4995 001 Neural Rendering for Computer Graphics
***weekly in class quiz dates
COURSEWORK
35% Programming Assignments (2)
15% Take Home Exam (1)
15% Written Homework (1)
25% Final Project
10% Quizzes (~weekly: 10 top scores count toward grade)
Grading Policy
98-100 A+ 4.33
93 – 97.9 A 4.00
90.0 – 92.9 A- 3.67
87-89.9 B+ 3.33
83-86.9 B 3.00
80-82.9 B- 2.67
77-79.9 C+ 2.33
73-76.9 C 2.00
70-72.9 C- 1.65
60-69.9 D 1.00
below 59.9 F 0.00
Attendance Policy, Class Expectations, and Make-Up Policy
Students are expected to attend class. Repeated non-excused absences (more than three) may impact your grade. Make-Up homework, projects and exams must be coordinated with the instructor. Excused absences must be consistent with your college’s Absence and Attendance Policy.
Guidelines for assignments
There is an emphasis on developing GPU programming skills while implementing concepts learned for real computer graphics applications. Students will complete 2 programming assignments and a final project using a GPU cluster. Implementing a neural volume renderer and then optimizing it for neural radiance fields, and implementing 3D Gaussian splatting are examples of past projects. The final project is designed by the student but must be approved by the instructor. Weekly quizzes are designed to help students access their understanding of course material on a regular basis. These quizzes also provide preparation for the written homework (one) and take-home exam (one) which are designed to help students develop problem solving skills that use mathematical concepts covered in the course material. The instructor will provide detailed assignment guidance/instructions with support material.
Late Policy:
Students are given five late days total for the course. Students can use late days without penalty for project one and/or two only. Late days cannot be used for the final project, homework, or take-home exam.
There is a 10% per day late deduction up to a maximum of a 50% reduction for late programming projects (after applied late days). Students are expected to submit all programming assignments (even if they are late).
To use the late days, you must notify the instructor in writing (email) by the assignment due date and receive a confirmation email from the instructor. When you submit your project, you must indicate the number of late days you are using.
You do not have to use your late days if you have a university allowed absence. See your college’s Excused Absence Policy.
The homework solutions are posted quickly as a study aid for the take-home exam.
***NO LATE HOMEWORK accepted except for a university allowed absence. ***
Columbia University Honor Code
In addition, all students must adhere to and uphold the honor codes of their respective colleges. All Columbia College and undergraduate Columbia Engineering students are committed to the following honor code:
I affirm that I will not plagiarize, use unauthorized materials, or give or receive illegitimate help on assignments, papers, or examinations. I will also uphold equity and honesty in the evaluation of my work and the work of others. I do so to sustain a community built around this Code of Honor.