GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Paper
  • Video
  • Shiny Dataset
  • Shiny Dataset Source
  • Real Dataset
  • Code

This view

  • Original page
  • New address

GeezerVison is showing dorverbin.github.io — Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields

Ref-NeRF: Structured View-Dependent
Appearance for Neural Radiance Fields
CVPR 2022 (Oral Presentation, Best Student Paper Honorable Mention)

Dor Verbin
Harvard University
Google Research 		Peter Hedman
Google Research 		Ben Mildenhall
Google Research
Todd Zickler
Harvard University 		Jonathan T. Barron
Google Research 		Pratul P. Srinivasan
Google Research

  • Paper

  • Video

  • Shiny Dataset

  • Shiny Dataset Source

  • Real Dataset

  • Code

Abstract

Neural Radiance Fields (NeRF) is a popular view synthesis technique that represents a scene as a continuous volumetric function, parameterized by multilayer perceptrons that provide the volume density and view-dependent emitted radiance at each location. While NeRF-based techniques excel at representing fine geometric structures with smoothly varying view-dependent appearance, they often fail to accurately capture and reproduce the appearance of glossy surfaces. We address this limitation by introducing Ref-NeRF, which replaces NeRF's parameterization of view-dependent outgoing radiance with a representation of reflected radiance and structures this function using a collection of spatially-varying scene properties. We show that together with a regularizer on normal vectors, our model significantly improves the realism and accuracy of specular reflections. Furthermore, we show that our model's internal representation of outgoing radiance is interpretable and useful for scene editing.

overview

Video

Reflection Direction Parameterization

Previous approaches directly input the camera's view direction into the MLP to predict outgoing radiance. We show that instead using the reflection of the view direction about the normal makes the emittance function significantly easier to learn and interpolate, greatly improving our results.

Integrated Directional Encoding

We explicitly model object roughness using the expected values of a set of spherical harmonics under a von Mises-Fisher distribution whose concentration parameter varies spatially:


We call this Integrated Directional Encoding, and we show experimentally that it allows sharing the emittance functions between points with different roughnesses. It also enables scene editing after training. Theoretically, our encoding is stationary on the sphere, similar to the Euclidean stationarity of NeRF's positional encoding.

Additional Synthetic Results

Results on Captured Scenes

Our method also produces accurate renderings and surface normals from captured photographs:

Scene Editing

We show that our structured representation of the directional MLP allows for scene editing after training. Here we show that we can convincingly change material properties.
We can increase and decrease material roughness:
We can also control the amounts of specular and diffuse colors, or change the diffuse color without affecting the specular reflections:

Citation

Acknowledgements

We would like to thank Lior Yariv and Kai Zhang for helping us evaluate their methods, and Ricardo Martin-Brualla for helpful comments on our text. DV is supported by the National Science Foundation under Cooperative Agreement PHY-2019786 (an NSF AI Institute, http://iaifi.org)
The website template was borrowed from Michaël Gharbi.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.