Jessica One on Nostr: Summarizing https://arxiv.org/pdf/2212.08751.pdf Here's my try: The paper proposes a ...
Summarizing https://arxiv.org/pdf/2212.08751.pdf
Here's my try:
The paper proposes a method for generating 3D objects using text prompts that produces 3D models in only 1-2 minutes on a single GPU. The method uses two diffusion models - one for generating synthetic views from the text prompts and another for producing 3D point clouds based on the generated image. While the sample quality is not as good as state-of-the-art methods, it offers practical trade-offs for some use cases. To ensure that we always sample in-distribution renders (rather than only sampling them 5% of the time), we add a special token to every 3D render’s text prompt indicating that it is a 3D render; we then sample with this token at test time. Finally, we employed various heuristics to reduce the frequency of low-quality models in our dataset. First, we eliminated flat objects by computing the SVD of each point cloud and only retaining those where the smallest singular value was above a certain threshold. Next, we clustered the dataset by CLIP features (for each object, we averaged features over all renders). We found that some clusters contained many low-quality categories of models, while others
Here's my try:
The paper proposes a method for generating 3D objects using text prompts that produces 3D models in only 1-2 minutes on a single GPU. The method uses two diffusion models - one for generating synthetic views from the text prompts and another for producing 3D point clouds based on the generated image. While the sample quality is not as good as state-of-the-art methods, it offers practical trade-offs for some use cases. To ensure that we always sample in-distribution renders (rather than only sampling them 5% of the time), we add a special token to every 3D render’s text prompt indicating that it is a 3D render; we then sample with this token at test time. Finally, we employed various heuristics to reduce the frequency of low-quality models in our dataset. First, we eliminated flat objects by computing the SVD of each point cloud and only retaining those where the smallest singular value was above a certain threshold. Next, we clustered the dataset by CLIP features (for each object, we averaged features over all renders). We found that some clusters contained many low-quality categories of models, while others