Deep Learning for Computer Graphics COMS BC3997 Spring 2024

Categories

OVERVIEW
This course covers neural network architectures (deep learning), related math and the fundamental theory and application of AI algorithms most popular in the field of computer graphics. Programming assignments will help students develop graphics processing unit (GPU) programming skills while implementing concepts learned in lectures and readings using deep learning APIs on a GPU cluster. Generative Adversarial Networks (GANs) for facial animation is a project example.

 

OBJECTIVES
This course covers the fundamental theory and application of AI algorithms in the context of computer graphics. The course begins with deep learning basics including related math review (numerical analysis and gradient optimization). Building upon this foundation, students learn essential deep learning concepts including: supervised, unsupervised and reinforcement learning, and operations relevant to neural network architectures (like backpropagation and fine tuning). There is an emphasis on developing GPU programming skills while implementing real computer graphics applications using learned concepts. Two homework assignments, weekly quizzes and one take-home exam compliment these programming assignments and help students evaluate their comprehension of course material. The course culminates with student-designed final projects that demonstrate creativity, and a depth of knowledge in two or more course topics. Convolutional neural networks for colorizing black and white movies, transformers for image classification, and generative adversarial networks for stylized rendering are application examples.

 

Instructor: Dr. Corey Toler-Franklin, Computer Science Department, Barnard College, Columbia University

Contact: Office Milstein 502B, ctolerfr@barnard.edu   ct3219@columbia.edu

Office Hours: TBD
Location:   203 Diana Center
Time: MW 11:40 am – 12:55 pm
 

Course Management: CourseWorks
 

PREREQUISITES:

Prerequisites: COMS W3157 Advanced Programming, Linear Algebra (UN2010), and Calculus I or higher.

**Contact instructor if you are not sure you are prepared for the course**

 

TEXTBOOK

Required

Title: Understanding Deep Learning

Author: Simon J.D. Prince
ISBN: 9780262048644
Published: December 5, 2023
Publisher: The MIT Press

 

SYLLABUS
COMPS-BC3997 Section 003 Deep Learning for Computer Graphics

Date
Topic
Reading & Media
Assignments
17-Jan
Introduction
22-Jan
Machine Learning Basics
Prince 1-1.2.1, 1.3, 2-2.2
course survey out
24-Jan***
Numerical Analysis, Gradient Optimization
Prince 6.1
course survey due
29-Jan
Cloud Computing & GPU Clusters | Neural Networks
Prince 3-3.5
31-Jan***
Neural Networks: Deep Networks
Prince 4-4.5
5-Feb
Neural Networks: Cost Functions, Regularization
Prince 5-5.7
7-Feb***
Neural Networks: Backpropagation | ML APIs
Prince 6.2, 6.5, 7.1-7.4
Proj.1 out
12-Feb
ML APIs: Python, Torch, CUDA, cuDNN, TensorFlow
14-Feb***
ML APIs | Training, Testing, Fine-tuning
Prince 6.3, 6.4, 7.5, 8-8.5, 9-9.3
19-Feb
Training, Testing, Fine-tuning
Proj.2 out
21-Feb***
Convolutional Neural Networks
Prince 10-10.5
Proj. 1 due
26-Feb
Final Project Discussion, Traditional Machine Learning
Final Project Guidelines out
28-Feb***
Autoencoders, Residual Networks
Prince 11-11.3
4-Mar
Transformers: Natural Language Processing,
Video Intro, Prince 12-12.4
Homework out
6-Mar***
Math for Computer Graphics
Proj 2. Due (Final Proj. Proposal due Mar. 8th)
11-Mar
Spring Break – NO CLASSES
13-Mar
Spring Break – NO CLASSES
18-Mar
Deep Learning – Inverse Graphics Problem: Current Trends
Homework due
20-Mar
Take Home Exam – No Class
Exam due Mar. 22nd
25-Mar
Generative Adversarial Networks
Prince 15-15.2, Song 2021
27-Mar
Reinforcement Learning
Prince 19-9.4, Mnih 2013, Hassan 2023
1-Apr
Neural Rendering
Pandey 2021
3-Apr
Image Denoising
Bako 2017
8-Apr
Learning from Physics, Motion from Video
Lerer 2016
Final Project Midpoint Evaluations
10-Apr
Autonomous Driving
Janai 2017, TED Talk
15-Apr
Robotics
Kim 2022, TED Talk
17-Apr
Deep Learning and Ethics
Prince 1.4, 21-21.8, TED Talk
22-Apr
TBD
24-Apr
Final Project Presentations
29-Apr
Final Project Presentations
Final Project due

 

***weekly in class quiz dates

Late Policy:

Students are given 5 late days total for the course. These can be used without penalty for project 1 and/or 2 only. They cannot be used for the final project, homework or take-home exam.

There is a 10% per day late deduction up to a maximum of a 50% reduction for late programming projects (after applied late days). It is the expectation that all programming assignments will be submitted, whether late or not.

To use the late days, you must notify the instructor in writing (email) by the assignment due date and receive a confirmation email from the instructor. When you submit your project, you must indicate the number of late days you are using.

You do not have to use your late days if you have a university allowed excuse and supporting documentation. See your college’s Excused Absence Policy.

The homework solutions are posted quickly as a study aid for the take-home exam.
***NO LATE HOMEWORK accepted except for a university allowed excuse. ***