Deep Learning for Computer Graphics COMS BC3168 Spring 2025

Categories

OVERVIEW
This undergraduate course covers deep learning basics, related math and the fundamental theory and application of AI algorithms that are popular in the field of computer graphics. Programming assignments will help students develop GPU programming skills while implementing concepts learned in lectures and readings using deep learning APIs on a GPU cluster. Convolutional neural networks (CNNs) for colorizing black and white movies is an example.

 

OBJECTIVES
This undergraduate course covers the fundamental theory and application of AI algorithms in the context of computer graphics. The course begins with deep learning basics including several lectures that cover related math (numerical analysis, gradient optimization, and math for computer graphics). Students will then learn fundamental deep learning concepts including supervised, unsupervised and reinforcement learning, modern neural network architectures like transformers, and operations relevant to neural network architectures like backpropagation and fine tuning. There is an emphasis on developing GPU programming skills while implementing real computer graphics applications using learned concepts. Two homework assignments, weekly quizzes and one take-home exam complement programming projects (which may be completed individually or in groups of 2-3), and are designed to help students develop problem solving skills that use mathematical concepts covered in class. These assignments also help students evaluate their comprehension of course material. The course culminates with student-designed final projects that demonstrate creativity, and a depth of knowledge in two or more course topics. Convolutional neural networks for colorizing black and white movies, transformers for image classification, and generative adversarial networks for stylized rendering are application examples.
 

Instructor: Dr. Corey Toler-Franklin, Computer Science Department, Barnard College, Columbia University

Contact: Office Milstein 502B, ct3219@columbia.edu

Office Hours: TBD
Location:  
Time: MW 11:40am-12:55pm
 

Course Management: CourseWorks
 

PREREQUISITES:

Prerequisites: CCOMS W3157 Advanced Programming, Linear Algebra (COMS W3251, APMA E3101, APMA E2101, MATH UN2010, or MATH UN2015), and Calculus I or higher.

**Contact instructor if you are not sure you are prepared for the course**

 

TEXTBOOK

Required

Title: Understanding Deep Learning

Author: Simon J.D. Prince
ISBN: 9780262048644
Published: December 5, 2023
Publisher: The MIT Press

 

SYLLABUS
COMS BC3168 001 Deep Learning for Computer Graphics

Date
Topic
Reading & Media
Assignments
22-Jan
Introduction
27-Jan
Machine Learning Basics
Prince 1-1.2.1, 1.3, 2-2.2
course survey out
29-Jan***
Numerical Analysis, Gradient Optimization
Prince 6.1
course survey due
3-feb
Cloud Computing & GPU Clusters | Neural Networks
Prince 3-3.5
5-feb***
Neural Networks: Deep Networks
Prince 4-4.5
10-Feb
Neural Networks: Cost Functions, Regularization
Prince 5-5.7
12-Feb***
Neural Networks: Backpropagation | ML APIs
Prince 6.2, 6.5, 7.1-7.4
Proj.1 out
17-Feb
ML APIs: Python, Torch, CUDA, cuDNN, TensorFlow
19-Feb***
ML APIs | Training, Testing, Fine-tuning
Prince 6.3, 6.4, 7.5, 8-8.5, 9-9.3
24-Feb
Training, Testing, Fine-tuning
Proj.2 out
26-Feb***
Convolutional Neural Networks
Prince 10-10.5
Proj. 1 due
3-Mar
Final Project Discussion, Traditional Machine Learning
Final Project Guidelines out
5-Mar***
Autoencoders, Residual Networks
Prince 11-11.3
10-Mar
Transformers: Natural Language Processing,
Video Intro, Prince 12-12.4
Homework out
12-Mar***
Math for Computer Graphics
Proj 2. Due (Final Proj. Proposal due Mar. 8th)
17-Mar
Spring Break – NO CLASSES
19-Mar
Spring Break – NO CLASSES
24-Mar
Deep Learning – Inverse Graphics Problem: Current Trends
Homework due
26-Mar
Take Home Exam – No Class
Exam due Mar. 22nd
31-Mar
Generative Adversarial Networks
Prince 15-15.2, Song 2021
2-Apr
Reinforcement Learning
Prince 19-9.4, Mnih 2013, Hassan 2023
7-Apr
Neural Rendering
Pandey 2021
9-Apr
Image Denoising
Bako 2017
14-Apr
Learning from Physics, Motion from Video
Lerer 2016
Final Project Midpoint Evaluations
16-Apr
Autonomous Driving
Janai 2017, TED Talk
21-Apr
Robotics
Kim 2022, TED Talk
23-Apr
Deep Learning and Ethics
Prince 1.4, 21-21.8, TED Talk
28-Apr
TBD
30-Apr
Final Project Presentations
5-May
Final Project Presentations
Final Project due

 

***weekly in class quiz dates

 

 

COURSEWORK

35% Programming Assignments (2)

15%  Take Home Exam (1)

15%  Written Homework (1)

25%  Final Project

10%  Quizzes (~weekly: 10 top scores count toward grade)

 

 

Grading Policy

98-100 A+ 4.33
93 – 97.9 A 4.00
90.0 – 92.9 A- 3.67
87-89.9 B+ 3.33
83-86.9 B 3.00
80-82.9 B- 2.67
77-79.9 C+ 2.33
73-76.9 C 2.00
70-72.9 C- 1.65
60-69.9 D 1.00
below 59.9 F 0.00

 

Attendance Policy, Class Expectations, and Make-Up Policy
Students are expected to attend class. Repeated non-excused absences (more than three) may impact your grade. Make-Up homework, projects and exams must be coordinated with the instructor. Excused absences must be consistent with your college’s Absence and Attendance Policy.

Guidelines for assignments
There is an emphasis on developing GPU programming skills while implementing concepts learned for real computer graphics applications. Students will complete two two-week long programming assignments and a final project (5 weeks) using a GPU cluster. Colorizing black and white photos and movies using a CNN, and implementing a diffusion model for generating photorealistic images are examples of projects. The final project is designed by the student but must be approved by the instructor. All projects may be completed individually, or in groups of up to three. Weekly quizzes are designed to help students access their understanding of course material on a regular basis. These quizzes also provide preparation for the written homework (one) and take-home exam (one) which are designed to help students develop problem solving skills that use mathematical concepts covered in the course material. The instructor will provide detailed assignment guidance/instructions with support material.

 

Late Policy:
Students are given five late days total for the course. Students can use late days without penalty for project one and/or two only. Late days cannot be used for the final project, homework, or take-home exam.

There is a 10% per day late deduction up to a maximum of a 50% reduction for late programming projects (after applied late days). Students are expected to submit all programming assignments (even if they are late). To use the late days, you must notify the instructor in writing (email) by the assignment due date and receive a confirmation email from the instructor. When you submit your project, you must indicate the number of late days you are using.

You do not have to use your late days if you have a university allowed absence. See your college’s Absence and Attendance Policy.

The homework solutions are posted quickly as a study aid for the take-home exam. Late homework is not accepted except for absences permitted by your college’s absence and attendance policy.

Honor Code
We, the students of Barnard College, resolve to uphold the honor of the College by engaging with integrity in all of our academic pursuits. We affirm that academic integrity is the honorable creation and presentation of our own work. We acknowledge that it is our responsibility to seek clarification of proper forms of collaboration and use of academic resources in all assignments or exams. We consider academic integrity to include the proper use and care for all print, electronic, or other academic resources. We will respect the rights of others to engage in pursuit of learning in order to uphold our commitment to honor. We pledge to do all that is in our power to create a spirit of honesty and honor for its own sake.
 

In addition, all students must adhere to and uphold the honor codes of their respective colleges. All Columbia College and undergraduate Columbia Engineering students are committed to the following honor code:
 

I affirm that I will not plagiarize, use unauthorized materials, or give or receive illegitimate help on assignments, papers, or examinations. I will also uphold equity and honesty in the evaluation of my work and the work of others. I do so to sustain a community built around this Code of Honor.

Wellness Statement
It is important for undergraduates to recognize and identify the different pressures, burdens, and stressors you may be facing, whether personal, emotional, physical, financial, mental, or academic. We as a community urge you to make yourself–your own health, sanity, and wellness–your priority throughout this term and your career here. Sleep, exercise, and eating well can all be a part of a healthy regimen to cope with stress. Resources exist to support you in several areas of your life, and we encourage you to make use of them. Should you have any questions about navigating these resources, please visit these sites:

http://barnard.edu/primarycare
https://barnard.edu/about-counseling
http://barnard.edu/wellwoman/about
https://www.health.columbia.edu/services/stressbusters

Center for Accessibility Resources & Disability Services (CARDS) Statement
If you believe you may encounter barriers to the academic environment due to a documented disability or emerging health challenges, please feel free to contact me and/or the Center for Accessibility Resources & Disability Services (CARDS). Any student with approved academic accommodations is encouraged to contact me during office hours or via email. If you have questions regarding registering a disability or receiving accommodations for the semester, please contact CARDS at (212) 854-4634, cards@barnard.edu, or learn more at barnard.edu/disabilityservices. CARDS is located in 101 Altschul Hall.

Affordable Access to Course Texts & Materials
We believe all students deserve to be able to access course texts. The high costs of textbooks and other course materials prohibit access and perpetuate inequity, and Barnard librarians and library staff are partnering with students, faculty, and staff to increase access. The purpose of this page is to highlight the steps the Barnard Library is taking to promote accessibility of texts on our campus.