Table of Contents
Shortcuts

Rendering

nerfacc.rendering(t_starts, t_ends, ray_indices=None, n_rays=None, rgb_sigma_fn=None, rgb_alpha_fn=None, render_bkgd=None)

Render the rays through the radience field defined by rgb_sigma_fn.

This function is differentiable to the outputs of rgb_sigma_fn so it can be used for gradient-based optimization. It supports both batched and flattened input tensor. For flattened input tensor, both ray_indices and n_rays should be provided.

Note

Either rgb_sigma_fn or rgb_alpha_fn should be provided.

Warning

This function is not differentiable to t_starts, t_ends and ray_indices.

Parameters:

  • t_starts (Tensor) – Per-sample start distance. Tensor with shape (n_rays, n_samples) or (all_samples,).

  • t_ends (Tensor) – Per-sample end distance. Tensor with shape (n_rays, n_samples) or (all_samples,).

  • ray_indices (Optional[Tensor]) – Ray indices of the flattened samples. LongTensor with shape (all_samples).

  • n_rays (Optional[int]) – Number of rays. Only useful when ray_indices is provided.

  • rgb_sigma_fn (Optional[Callable]) – A function that takes in samples {t_starts, t_ends, ray indices} and returns the post-activation rgb (…, 3) and density values (…,). The shape … is the same as the shape of t_starts.

  • rgb_alpha_fn (Optional[Callable]) – A function that takes in samples {t_starts, t_ends, ray indices} and returns the post-activation rgb (…, 3) and opacity values (…,). The shape … is the same as the shape of t_starts.

  • render_bkgd (Optional[Tensor]) – Background color. Tensor with shape (3,).

Returns:

Ray colors (n_rays, 3), opacities (n_rays, 1), depths (n_rays, 1) and a dict containing extra intermediate results (e.g., “weights”, “trans”, “alphas”)

Return type:

Tuple[Tensor, Tensor, Tensor, Dict]

Examples:

>>> t_starts = torch.tensor([0.1, 0.2, 0.1, 0.2, 0.3], device="cuda:0")
>>> t_ends = torch.tensor([0.2, 0.3, 0.2, 0.3, 0.4], device="cuda:0")
>>> ray_indices = torch.tensor([0, 0, 1, 1, 1], device="cuda:0")
>>> def rgb_sigma_fn(t_starts, t_ends, ray_indices):
>>>     # This is a dummy function that returns random values.
>>>     rgbs = torch.rand((t_starts.shape[0], 3), device="cuda:0")
>>>     sigmas = torch.rand((t_starts.shape[0],), device="cuda:0")
>>>     return rgbs, sigmas
>>> colors, opacities, depths, extras = rendering(
>>>     t_starts, t_ends, ray_indices, n_rays=2, rgb_sigma_fn=rgb_sigma_fn)
>>> print(colors.shape, opacities.shape, depths.shape)
torch.Size([2, 3]) torch.Size([2, 1]) torch.Size([2, 1])
>>> extras.keys()
dict_keys(['weights', 'alphas', 'trans'])