Really great course. I loved it. Learned so many new things. Professor and TAs are also super nice.

I truly and deeply loved this course, especially since it somewhat converges with 3D Computer Vision through differentiable and inverse rendering.

The Content

This course covers a lot of material: ray tracing, signed distance functions, volume rendering, radiance fields, Gaussian splatting, differentiable and inverse rendering, physics simulation, and generative models, from classical, differentiable and neural approaches. Some of these topics are so complex that one only touches the intuition and surface details (NeRF and 3D Gaussian Splatting), and implements lighter versions of the techniques.

The Lectures

There are too many of them, and sometimes a bit too lengthy. Some weeks require 4 hours of lectures, and some others even 7. Just for lectures. This is not necessarily a complaint, since they are very detailed, very well explained, and very thorough, while still being entertaining to watch. On the bad side, you need to actually be interested in the subject to watch them all since the lectures cover a lot more material than the assignments, and if you're the kind of person who only studies what's covered in the assignments you'll end up watching/learning less than half the content.

The Assignments

There are 8 assignments divided in 6 topics. Each assignment is worth 8% of the final grade (64% total), and basically anyone can work on them whether they're enrolled or not since the assignments' site and their repo are publicly available. You can also frontload since all the assignments are released from day one; you'll only need to keep track of submission dates.

• RAY TRACING is about implementing the fundamental parts of a Whitted ray tracer. For people who took CG this feels like a review while still adding bits not covered in that course. For other people, this one might be the most time-consuming assignment since it asks you to implement many algorithms not very complex but not necessarily trivial.

• SIGNED DISTANCE FUNCTIONS is about implementing implicit SDF primitives and simple CSG operations. This one is actually very easy and straightforward. The most interesting part is implementing the sphere tracing loop; something not required in the CG course.

• NEURAL IMPLICIT SURFACES is the complement of the above, where one trains a SIREN network that learns to represent SDFs from input point clouds (actually, sampled meshes). The main challenge comes from accurately translating formulas to PyTorch.

• VOLUMETRIC RENDERING is about implementing ray marching on a couple of density-based pre-defined volumes. This topic is also covered in the CG course but only at the lecture level; here one actually implements it. The hardest part is not the actual assignment but all the theory behind light, photometry and radiometry, which is quite dense.

• NEURAL RADIANCE FIELDS is the complement of the above, where one implements and trains a tiny NeRF that fits in a GLSL fragment shader. Here you are given A LOT of starter code since the network is very complex to create from end to end.

• 2D GAUSSIAN SPLATTING is a deeply simplified exercise on Gaussian Splatting that kinda feels like cheating, since full 3D Gaussian Splatting is also fairly intimidating. This whole assignment feels like implementing only one of the many steps in 3DGS. Still, the actual assignment is kinda fun and visually pleasing.

• XPBD is about classical physics simulation based on particle positions where one must define a set of constraints and their gradients. After watching the relevant lectures I think I finished this one in around half an hour.

• DIFFUSION MODELS is about implementing DDPM from scratch in PyTorch and GLSL. Really easy if one took DL before, although that's not necessary.

The particularly interesting part of the assignments is the CREATIVE EXPRESSION sections that all assignments have, where we can do whatever we want that builds on top of everything and anything covered so far. Even if the core assignments could take you less than 2 hours each, if you're really interested in the course, this could take you over 10-15 hours just for 1 or 2 points this is worth. I'd argue that this is the most important part of the assignments and the course: the opportunity to go above and beyond mostly for ourselves (and optionally for others to see).

The Midterm

Really easy; as easy as the problem set. Here you'll solve a few exercises by hand, and step by step, and you'll upload your handwritten results. This might be the weakest aspect of the course; more of an excuse to review the material covered so far (which is still not bad).

The Final Project

As lovely as the Creative Expression sections. You can do whatever you want, creative or technical, as long as you cover at least one topic from the course (except ray tracing; that's basically a review and not the core of the course). Here you'll notice just how creative or inventive some people might be. This term there were many beautiful projects and at least one really awesome that displayed the deepest dedication.

Overall

All in all, I'm still undecided on whether this might be my favorite OMSCS course or not, only competing with RAIT, which was a really fun course, even if somewhat easier than this one. Still an amazing and well-crafted course that I'll love reviewing from time to time just for the sake of watching these awesome lectures once again.

Deeply recommend it. You don't need to take Deep Learning of Computer Graphics to follow along but they're quite complementary with not much overlap.

Lectures

The lectures were super clear, especially with all the visuals present throughout. Bo Zhu really did a great job breaking down complex ideas into chunks that actually made sense. The way he structured everything was engaging and the material felt relevant and current. I found myself rewatching lectures because they're so easy to come back to.

Assignments

The programming assignments had a creative side to them, which I loved. They're structured as Jupyter notebooks and/or fragment shader files using an app the staff built. You open the relevant files and implement the code sections marked "write code here." You can go as far as you want with the creative parts, and there's room to explore more if you get into it. As someone who's naturally creative, I poured genuine effort into them.

Exams

There was just one midterm, which was manageable and open book. For the final, you got to choose between doing a creative render or writing a technical report, so you had some flexibility there.

Staff Engagement

Bo and the TAs were very present the entire semester from start to finish. Questions on Ed were answered quickly, replies were always friendly, and the support was genuine throughout.

Overall

I have absolutely no regrets taking this. It was a super fun class and I loved the whole experience. If you're thinking about it, I'd really encourage you to take it. :)