COS 598a: Machine Learning-Driven Video Systems, Spring 2022

Instructor: Ravi Netravali
Lectures: Monday/Wednesday 9:30am-10:50am (ET) in Computer Science Building, Room 302
Office Hours: by appointment (

Course Overview

Video applications are pervasive, ranging from streaming and virtual reality to real-time conferencing and analytics. As these applications evolve, however, they face an increasing tension between the rich experiences and strict performance objectives that users demand. For instance, many services require increasingly data-dense videos or high-accuracy (expensive) neural networks for analytics, but must operate smoothly in resource-constrained mobile settings. This graduate, research-driven seminar explores a range of systems and machine learning optimizations that improve the performance, efficiency, and robustness of modern video applications. The course will be heavily based on studying, presenting, and actively discussing research papers, and will also involve a semester long research project.


Lectures will take place during the scheduled slots (i.e., 9:30-10:50am on Monday/Wednesday) every week, and students are expected to attend and actively participate. We are currently slated to have lectures in person. However, we will monitor and respond to the evolving COVID-19 situtation and corresponding university policies. In the event that courses are moved to remote instruction, Zoom information and protocols will be posted here.


  • 40% Participation in paper discussions

  • 25% Paper presentation/lecture

  • 35% Research project (report and presentation)

Paper Reading and Discussion

A major component of this course is reading and discussing research papers in depth. To ensure a lively and focused discussion, you should closely read each paper and add comments and questions on Perusall by 5pm ET the night before the corresponding lecture. Please plan to provide at least five comments or questions for each paper and also follow the comments from the other students and the course staff. In addition, please come to class prepared with several points that will substantially contribute to the group discussion. General tips on reading research papers can be found here, and we strongly encourage you to review and follow the discussion guidelines/suggestions from last year's COS 561 offering. Your participation grade will be determined based on attendance and, more importantly, substantial contributions to paper discussions both on Perusall and in class.

Class attendance is required. However, we understand that students may be facing a variety of challenging circumstances this semester. If you are unable to attend a lecture, please let the instructor know as soon as possible so that we can best accomodate the absence.

Paper Presentation/Lecture

In each class, one student will be expected to present the scheduled paper and lead the discussion for it. Presentations should start with a (roughly) 30-minute overview of the paper. The format of this part of the presentation should be ‘‘conference style,’’ i.e., covering the domain and relevant background for the paper, the problem statement and challenges, the solution, results, and potential limitations and improvements. However, the presentation should go into more detail than a typical conference talk would, particularly on the design of the proposed solution; for this reason, while public conference slides for the paper can be used as an aid, they will not suffice for the lecture. The remainder of the lecture will involve leading discussion by: (1) fielding questions, (2) posing questions, and (3) covering common questions and misconceptions based on Perusall comments from the class. Non-presenters are expected to actively participate in the discussions and bring discussion points (including questions) of their own. Active participation will lead to a lively discussion that will benefit everyone.

Research Project

In addition to paper reading, this course will also include a semester-long research project. Students will carry out projects individually or in pairs. The goal of this research project is not necessarily to fully implement a research idea. Instead, students are encouraged to pick a problem that is new (i.e., previously unsolved) and exciting to them, and focus primarily on building (small-scale) prototypes and collecting measurements to motivate the problem and their solution. Thus, implementation is a key aspect of the project, but students are encouraged to aim high, and not feel restricted to topics or ideas that could be 100% implemented before the course concludes. The scope of acceptable topics is quite large – anything related to video systems and ML video analysis, transmission, storage, and processing are fair game. Extensions to ongoing research projects can be used if in scope; please see the instructor to discuss your specific ongoing project and how you would like to extend it for the course. It is strongly encouraged to begin thinking about project topics early on in the semester by reviewing the reading list/topics, and discussing with the instructor. The timeline and deliverables for the project are:

  • Team formation (due 2/4 at 5pm ET)

  • Meet with instructor to discuss proposed project idea (in class on 2/28)

  • 1-2 page project proposal+plan (due 3/16 at 5pm ET)

  • Final project presentation (in class on 4/18 and 4/20)

  • 5-6 page final project report (due Dean's date: 5/3 at 5pm ET); this should be submitted as a PDF generated using the Usenix conference research paper format.