ml-for-3d-course/main/en/unit3/introduction.md

Introduction

ML-friendly 3D

Let's take a step back and consider the generative 3D pipeline as a whole.

After multi-view diffusion comes ML-friendly 3D. This is some non-mesh representation of 3D that's easy for AI to handle.

In the current 3D research ecosystem, this can be a lot of things:

• Gaussian Splatting: Detailed in this unit
• Triplanes: Latest state-of-the-art. Learn more in this Community Notebook
• NeRFs: Synthesize novel views with a neural network
• and more

This is all changing very rapidly. So, like in the case of multi-view diffusion, ML-friendly 3D can be treated like a black box, using pre-trained models.

Gaussian Splatting

In this unit, I'll be diving deeper into one of these: Gaussian Splatting.

The reason I'm diving deeper into this is that, unlike the other representations, splats can be rendered in real-time, making them suitable for end-to-end 3D applications where everything is AI-compatible.

Let's get started!