CAP5404 Deep Learning for Computer Graphics Fall 2022

Categories

OVERVIEW

This course covers fundamental theory and application of popular artificial intelligence (AI) algorithms in computer graphics. Introduces several neural network architectures and the mathematical principles behind them. A semester-long project motivated by research publications teaches technical writing and graphics processing unit (GPU) programming on a GPU cluster. Convolutional neural networks for denoising movies and generative adversarial networks for animation are project examples.

 

 

OBJECTIVES

This course teaches students the mathematical principles behind deep learning AI algorithms and how to implement them to solve research problems in computer graphics. Concepts presented include: supervised, unsupervised and reinforcement learning; neural network (deep learning) architectures including convolutional neural networks (CNNs) and autoencoders; and related algorithms and techniques like backpropagation and fine-tuning. Topics presented in this graduate level course are explored through a semester-long project where students use GPU programming to implement theories and concepts they have learned in the course. Some project examples include, CNNs for denoising movie frames, and generative adversarial networks (GANs) for facial animation. Several lectures focus on technical paper writing and presentation skills. In the second half of the course, students read research papers that incorporate deep learning concepts in the context of computer graphics research. Weekly quizzes are designed to help students gauge their understanding of course material. These quizzes prepare students for the written homework assignment and take-home exam which are designed to develop problem solving skills using mathematical concepts covered in the course material.

 

Learning Objectives:
Students will learn fundamental concepts for solving engineering problems related to deep learning. They will apply mathematical concepts to develop

  • AI algorithms in a programming project. Students will gain experience with GPU programming.

    Expected Outcomes:
    1. An ability to identify, formulate, and solve complex engineering problems by applying principles of engineering, science, and mathematics.
    2. An ability to communicate effectively with a range of audiences.
    3. An ability to function effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives.
    4. An ability to develop and conduct appropriate experimentation, analyze and interpret data, and use engineering judgment to draw conclusions.
    5. An ability to acquire and apply new knowledge as needed, using appropriate learning strategies.

    Instructor: Dr. Corey Toler-Franklin, CISE Department, University of Florida

    Contact: Office CSE 332 or  Lab CSE 319, ctoler@cise.ufl.edu

    Office Hours: TBD

    Location:  Zoom (See details in canvas)

    Time: MWF Period 5 (11:45 am – 12:35 pm)

    Course Management: Canvas

    Website:  http://www.corey.toler-franklin.com/course/deep-learning-for-computer-graphics-cap5404-fall-2022/

     

    PREREQUISITES:

    Proficiency in a programming Language (Python and/or C++ recommended), Data Structures and Algorithms, Linear Algebra, and Calculus.

    **Contact instructor if you are not sure you are prepared for the course**

    TEXTBOOK

    Required

     

    ISBN: 9780262035613

    Author: Ian Goodfellow, Yoshua Bengio and Aaron Courville,


    Title: Deep Learning

    Free online version
    Online

     




    [/vc_accordion]

    SYLLABUS

    CAP5404 Spring2022 Syllabus
    Date
    Topic
    Reading
    Assignments
    24-Aug
    Introduction
    course survey out
    26-Aug
    Machine Learning Basics
    Goodfellow 5-5.2.0, 5.3
    29-Aug
    Machine Learning Basics
    course survey due
    31-Aug
    HiPerGator Training -In-Person + Zoom
    2-Sep ***
    Neural Networks
    Goodfellow 6, 6.1, 6.4
    5-Sep
    Labor Day: No Class
    7-Sep
    Neural Networks
    Goodfellow 4.3, 5.9, 6.5
    9-Sep ***
    Course Project Discussion|Neural Networks
    Goodfellow 7.12, 8.4, 8.7.1
    course proj. out
    12-Sep
    Neural Networks
    Goodfellow 14, 14.1, 14.9
    course proj. part1 out
    14-Sep
    Neural Networks
    16-Sep ***
    Python, Torch, CUDA, cuDNN, TensorFlow
    19-Sep
    Python, Torch, CUDA, cuDNN, TensorFlow
    21-Sep
    Training, Testing, Fine-tuning
    Goodfellow 15.2, 7.4
    23-Sep ***
    Training, Testing, Fine-tuning
    26-Sep
    Convolutional Neural Networks
    Goodfellow 9-9.3
    28-Sep
    Convolutional Neural Networks
    course proj. part2 out
    30-Sep ***
    Traditional Machine Learning
    Toler-Franklin 2010
    course proj. part1 due
    3-Oct
    Transformers, Recurrent Neural Networks
    Goodfellow 10-10.2.2, 10.10.1
    5-Oct
    No Class: Deep Learning in Medicine
    Zhang, Heldermon &Toler-Franklin 2020
    7-Oct ***
    Home Coming
    10-Oct
    Natural Language Processing
    12-Oct
    Deep Learning in Graphics: Inverse Graphics
    14-Oct ***
    Deep Learning in Graphics: Recent trends
    17-Oct
    Generative Adversarial Networks
    Goodfellow2014
    course proj. part3 out
    19-Oct
    Generative Adversarial Networks
    Song 2021, Gal 2021, Li 2021
    course proj.part2. due
    21-Oct ***
    Reinforcement Learning
    course proj. proposal
    24-Oct
    Reinforcement Learning
    Mnih2013, Peng 2018
    26-Oct
    Technical Writing| Discussion Course Proj.
    28-Oct ***
    Technical Writing| Discussion Course Proj.
    31-Oct
    Technical Writing| Discussion Course Proj.
    2-Nov
    Image Synthesis
    Pandey 2021
    written hw out
    4-Nov
    TBD
    7-Nov ***
    Image Denoising
    Bako2017
    9-Nov
    Motion from Video
    Karpathy 2014
    11-Nov
    Veteran’s Day: No Class
    course proj.mid eval
    14-Nov
    Motion from Video
    Habibie 2022
    written hw due
    16-Nov ***
    Take Home Exam
    Take Home Exam
    18-Nov
    Learning from Physics
    Lerer2016
    21-Nov
    Autonomous Driving
    23-Nov
    Thanksgiving: No Class
    25-Nov
    Thanksgiving: No Class
    28-Nov
    Autonomous Driving
    Janai 2017, Yuan 2022
    30-Nov
    Robotics
    TED Talk 2.
    2-Dec
    Robotics
    Pinto 2017
    5-Dec
    TBD
    7-Dec
    TBD
    course proj.part3. due

     

    ***weekly in class quiz dates

    COURSEWORK

    All assignments are distributed and submitted in Canvas.

    60% Course Project (submitted in three parts)

    15%  Take Home Exam (1)

    15%  Written Homework (1)

    10%  Quizzes (~weekly)

     

    Attendance Policy, Class Expectations, and Make-Up Policy
    Attendance is expected and noted. One half of a letter grade will be deducted (e.g. an A becomes a B+) for missing more than 3 classes for the semester without a documented university excused absence. Make-Up homework, projects and exams will be coordinated with the instructor for university excused absences. Excused absences must be consistent with university policies in the graduate catalog (http://gradcatalog.ufl.edu/content.php?catoid=10&navoid=2020#attendance) and require appropriate documentation.

     

    Grading Policy
    93.4 – 100 A 4.00
    90.0 – 93.3 A- 3.67
    86.7 – 89.9 B+ 3.33
    83.4 – 86.6 B 3.00
    80.0 – 83.3 B- 2.67
    76.7 – 79.9 C+ 2.33
    73.4 – 76.6 C 2.00
    70.0 – 73.3 C- 1.67
    66.7 – 69.9 D+ 1.33
    63.4 – 66.6 D 1.00
    60.0 – 63.3 D- 0.67
    0 – 59.9 E 0.00

    More information on UF grading policy may be found at:
    http://gradcatalog.ufl.edu/content.php?catoid=10&navoid=2020#grades

    Late Policy: Late programming projects will receive a late penalty of 10% per day late up to a maximum of a 50% reduction unless there is a documented university excused absence. Students are permitted 1 free late pass for 1 programming assignment (not including the final project which is due at the end of the semester). No late penalties will be applied for up to 1 week over the deadline when using a late pass. The written homework is reviewed in class in preparation for the exam and cannot be turned in late without a documented university excused absence.

     

    Students Requiring Accommodations
    Students with disabilities who experience learning barriers and would like to request academic accommodations
    should connect with the disability Resource Center by visiting https://disability.ufl.edu/students/get-started/. It is
    important for students to share their accommodation letter with their instructor and discuss their access needs, as
    early as possible in the semester.
    Course Evaluation
    Students are expected to provide professional and respectful feedback on the quality of instruction in this course by
    completing course evaluations online via GatorEvals. Guidance on how to give feedback in a professional and
    respectful manner is available at https://gatorevals.aa.ufl.edu/students/. Students will be notified when the
    evaluation period opens, and can complete evaluations through the email they receive from GatorEvals, in their
    Canvas course menu under GatorEvals, or via https://ufl.bluera.com/ufl/. Summaries of course evaluation results
    are available to students at https://gatorevals.aa.ufl.edu/public-results/.

     

    In-Class Recording
    Students are allowed to record video or audio of class lectures. However, the purposes for which these recordings
    may be used are strictly controlled. The only allowable purposes are (1) for personal educational use, (2) in
    connection with a complaint to the university, or (3) as evidence in, or in preparation for, a criminal or civil
    proceeding. All other purposes are prohibited. Specifically, students may not publish recorded lectures without the
    written consent of the instructor.

     

    A “class lecture” is an educational presentation intended to inform or teach enrolled students about a particular
    subject, including any instructor-led discussions that form part of the presentation, and delivered by any instructor
    hired or appointed by the University, or by a guest instructor, as part of a University of Florida course. A class
    lecture does not include lab sessions, student presentations, clinical presentations such as patient history,
    academic exercises involving solely student participation, assessments (quizzes, tests, exams), field trips, private
    conversations between students in the class or between a student and the faculty or lecturer during a class session.
    Publication without permission of the instructor is prohibited. To “publish” means to share, transmit, circulate,
    distribute, or provide access to a recording, regardless of format or medium, to another person (or persons),
    including but not limited to another student within the same class section. Additionally, a recording, or transcript
    of a recording, is considered published if it is posted on or uploaded to, in whole or in part, any media platform,
    including but not limited to social media, book, magazine, newspaper, leaflet, or third party note/tutoring services.
    A student who publishes a recording without written consent may be subject to a civil cause of action instituted by
    a person injured by the publication and/or discipline under UF Regulation 4.040 Student Honor Code and Student
    Conduct Code.
    University Honesty Policy
    UF students are bound by The Honor Pledge which states, “We, the members of the University of Florida community,
    pledge to hold ourselves and our peers to the highest standards of honor and integrity by abiding by the Honor Code.
    On all work submitted for credit by students at the University of Florida, the following pledge is either required or
    implied: “On my honor, I have neither given nor received unauthorized aid in doing this assignment.” The Conduct
    Code (https://sccr.dso.ufl.edu/process/student-conduct-code/) specifies a number of behaviors that are in violation
    of this code and the possible sanctions. If you have any questions or concerns, please consult with the instructor or
    TAs in this class.

     

    Commitment to a Safe and Inclusive Learning Environment

    The Herbert Wertheim College of Engineering values broad diversity within our community and is committed to
    individual and group empowerment, inclusion, and the elimination of discrimination. It is expected that every
    person in this class will treat one another with dignity and respect regardless of gender, sexuality, disability, age,
    socioeconomic status, ethnicity, race, and culture.
    If you feel like your performance in class is being impacted by discrimination or harassment of any kind, please
    contact your instructor or any of the following:
    • Your academic advisor or Graduate Program Coordinator
    • Jennifer Nappo, Director of Human Resources, 352-392-0904, jpennacc@ufl.edu
    • Curtis Taylor, Associate Dean of Student Affairs, 352-392-2177, taylor@eng.ufl.edu
    • Toshikazu Nishida, Associate Dean of Academic Affairs, 352-392-0943, nishida@eng.ufl.edu
    Software Use
    All faculty, staff, and students of the University are required and expected to obey the laws and legal agreements
    governing software use. Failure to do so can lead to monetary damages and/or criminal penalties for the individual
    violator. Because such violations are also against University policies and rules, disciplinary action will be taken as
    appropriate. We, the members of the University of Florida community, pledge to uphold ourselves and our peers to
    the highest standards of honesty and integrity.
    Student Privacy
    There are federal laws protecting your privacy with regards to grades earned in courses and on individual
    assignments. For more information, please see: https://registrar.ufl.edu/ferpa.html
    Campus Resources:
    Students will have access to a GPU cluster for completing the course project.

    Campus Resources:
    Students will have access to the HiPerGator GPU cluster for completing the course project.
    Health and Wellness
    U Matter, We Care:
    Your well-being is important to the University of Florida. The U Matter, We Care initiative is committed to
    creating a culture of care on our campus by encouraging members of our community to look out for one another
    and to reach out for help if a member of our community is in need. If you or a friend is in distress, please contact
    umatter@ufl.edu so that the U Matter, We Care Team can reach out to the student in distress. A nighttime and
    weekend crisis counselor is available by phone at 352-392-1575. The U Matter, We Care Team can help connect
    students to the many other helping resources available including, but not limited to, Victim Advocates, Housing
    staff, and the Counseling and Wellness Center. Please remember that asking for help is a sign of strength. In case
    of emergency, call 9-1-1.
    Counseling and Wellness Center: https://counseling.ufl.edu, and 392-1575; and the University Police
    Department: 392-1111 or 9-1-1 for emergencies.
    Sexual Discrimination, Harassment, Assault, or Violence
    If you or a friend has been subjected to sexual discrimination, sexual harassment, sexual assault, or violence
    contact the Office of Title IX Compliance, located at Yon Hall Room 427, 1908 Stadium Road, (352) 273-1094,
    title-ix@ufl.edu
    Sexual Assault Recovery Services (SARS)
    Student Health Care Center, 392-1161.
    University Police Department at 392-1111 (or 9-1-1 for emergencies), or http://www.police.ufl.edu/.
    COVID-19
    • You are expected to wear approved face coverings at all times during class and within buildings even if
    you are vaccinated.
    • If you are sick, stay home and self-quarantine. Please visit the UF Health Screen, Test & Protect website
    about next steps, retake the questionnaire and schedule your test for no sooner than 24 hours after your
    symptoms began. Please call your primary care provider if you are ill and need immediate care or the UF
    Student Health Care Center at 352-392-1161 (or email covid@shcc.ufl.edu) to be evaluated for testing and
    to receive further instructions about returning to campus.
    • If you are withheld from campus by the Department of Health through Screen, Test & Protect, you are not
    permitted to use any on campus facilities. Students attempting to attend campus activities when withheld
    from campus will be referred to the Dean of Students Office.
    • UF Health Screen, Test & Protect offers guidance when you are sick, have been exposed to someone who
    has tested positive or have tested positive yourself. Visit the UF Health Screen, Test & Protect website for
    more information.
    • Please continue to follow healthy habits, including best practices like frequent hand washing. Following
    these practices is our responsibility as Gators.
    Academic Resources
    E-learning technical support, 352-392-4357 (select option 2) or e-mail to Learning-support@ufl.edu.
    https://lss.at.ufl.edu/help.shtml.
    Career Resource Center, Reitz Union, 392-1601. Career assistance and counseling; https://career.ufl.edu.
    Library Support, http://cms.uflib.ufl.edu/ask. Various ways to receive assistance with respect to using the
    libraries or finding resources.
    Teaching Center, Broward Hall, 392-2010 or 392-6420. General study skills and tutoring.
    https://teachingcenter.ufl.edu/.
    Writing Studio, 302 Tigert Hall, 846-1138. Help brainstorming, formatting, and writing papers.
    https://writing.ufl.edu/writing-studio/.
    Student Complaints Campus: https://sccr.dso.ufl.edu/policies/student-honor-code-student-conductcode/;https://care.dso.ufl.edu.
    On-Line Students Complaints: http://www.distance.ufl.edu/student-complaint-process.

  • Leave a Reply

    Your email address will not be published.