NeRFAcc

Packages for accelerating rendering by skipping empty voxels.

ESTIMATORS

Create a occupancy grid

# Create an occupancy grid
estimator = nerfacc.OccGridEstimator(aabb,
                                     resolution=128,
                                     levels=1 # Num levels of grid
                                    )

# NOTE: the occupancy and binary grids are set to all 0s
# NOT ALL 1s !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Update the occupancy grid

# Thus, we must do this update at the begining to avoid
# skipping all voxels
estimator.update_every_n_steps(step,
                               eval_fn_occ,
                               occ_thre=0.01,
                               ema_decay=0.95,
                               warmup_steps=256,
                               n=16
                              )
# Step & n:
if step % n == 0 and self.training:
  do update

# warmup_steps:
if step < warmup_steps: 
    eval occupancy of all voxels + update
else:
  eval ~1/4 voxels

# ema_decay: !!!
# This means the occ will become smaller if it has not been 
# updated for a long time...
self.occs[cell_ids] = torch.maximum(self.occs[cell_ids] * ema_decay, 
                                    occ_eval)

# occ_thre:
# max val to determine whether it is occupied
thre = torch.clamp(self.occs[self.occs >= 0].mean(), max=occ_thre)
self.binaries = (self.occs > thre).view(self.binaries.shape)

Sampling

# This function would skip empty space based on occ grid
# NOTE: sigma/alpha is for early termination 
estimator.sampling(rays_o, # (N, 3)
                   rays_d, # (N, 3) 
                   sigma_fn=None, 
                   alpha_fn=None, 
                   near_plane=0.0, # near plane
                   far_plane=10000000000.0, # far plane 
                   t_min=None, # min distance (N, )
                   t_max=None, # max distance (N, ) 
                   render_step_size=0.001,
                   early_stop_eps=0.0001, 
                   alpha_thre=0.0, 
                   stratified=False, # rand noise
                   cone_angle=0.0)


# early_stop_eps:
vis = transmittance >= early_stop_eps
# alpha_thre:
vis = vis & (alphas >= alpha_thre)

Summary

Occupancy grid is used to skip empty space. At the beginning, mean occ will be the thresh for empty space. After training, mean occ is likely larger than thresh, then thresh will be used for empty space. The update occupancy grid has ema decay. That is said, if a voxel has small occ for many steps, it will be pruned. (May be set thresh to 0 at the first several steps would be a good choice?)

Parameters in sampling are primarily used for early termination of the sampling.