SO-NeRF
Active View Planning for NeRF using Surrogate Objectives
New York University
Coming Soon!
Coming Soon!

Overview of SO-NeRF against a baseline, ActiveNeRF. (a) Given any training mesh, we compute the individual surrogate objective scores C, Q, D and T, (b) train a model, SOARNet (c) for trajectory planning, without the need of prior visit of the candidate poses (unlike ActiveNeRF). (d) We can achieve comparable NeRF quality, but with a significant speed-up compared to the baseline.
Abstract
Despite the great success of Neural Radiance Fields (NeRF), its data-gathering process remains vague with only a general rule of thumb of ``sampling as densely as possible''. The lack of understanding of what actually constitutes good views for NeRF makes it difficult to actively plan a sequence of views that yield the maximal reconstruction quality. We propose Surrogate Objectives for Active Radiance Fields (SOAR), which is a set of interpretable functions that evaluates the goodness of views using geometric and photometric visual cues - surface coverage, geometric complexity, textural complexity, and ray diversity. Moreover, by learning to infer the SOAR scores from a deep network, SOARNet, we are able to effectively select views in mere seconds instead of hours, without the need for prior visits to all the candidate views or training any radiance field during such planning. Our experiments show SOARNet outperforms the baselines with
SOAR Score Generation Pipeline

The overall pipeline of our proposed approach. (a) Given an object, (b) we initialize via pseudo-coverage initialization to obtain a set of starting poses
SO-NeRF vs ActiveNeRF
Training Budget,
Citation
If you use this work or find it helpful, please consider citing: (bibtex)
@article{lee2023so, title={SO-NeRF: Active View Planning for NeRF using Surrogate Objectives}, author={Lee, Keifer and Gupta, Shubham and Kim, Sunglyoung and Makwana, Bhargav and Chen, Chao and Feng, Chen}, journal={arXiv preprint arXiv:2312.03266}, year={2023} }