Nerf.PCG_MULT
— ConstantImplementation of the PCG32 RNG.
We use stateless implementation as it is not allowed to pass mutable structs inside to CUDA kernels. So instead we pass only the initial state to the kernel.
From "PCG: A Family of Simple Fast Space-Efficient Statistically Good Algorithms for Random Number Generation" (2014) https://www.pcg-random.org/index.html
Nerf.BBox
— TypeAxis-aligned bounding box.
Parameters:
min::SVector{3, Float32}
: Minimum point of the bbox.max::SVector{3, Float32}
: Maximum point of the bbox.
Nerf.BBox
— MethodBBox(b::SVector{3, Float32})
Create BBox that encloses a point in 3D space.
Arguments:
b::SVector{3, Float32}
: 3D point.
Nerf.BBox
— MethodBBox()
Create an empty/invalid bbox. By setting min
and max
points to ∞
7 -∞
we ensure that all operations on the empty bbox give correct results.
Nerf.RayBundle
— TypeTODO
NerfUtils.CameraIntrinsics
— MethodCameraIntrinsics(width, height, fov)
Construct intrinsics for synthetic dataset.
NerfUtils.CameraIntrinsics
— MethodCameraIntrinsics(config)
Construct CameraIntrinsics from a config file.
Base.intersect
— MethodBase.intersect(b1::BBox, b2::BBox)
Compute intersection of the two bounding boxes.
Base.intersect
— MethodBase.intersect(b::BBox, r::Ray)
Compute intersection of bounding box with the ray.
Returns:
tmin
, tmax
ray parameters. To obtain actual intersection points, compute point along the ray: ray(tmin)
& ray(tmax)
.
In case when ray does not intersect BBox at all, returns (Inf32, Inf32)
.
Base.union
— MethodBase.union(b1::BBox, b2::BBox)
Compute union of the two bounding boxes.
Nerf.advance
— Functionadvance(state::UInt64, δ)
Multi-step advance function (jump-ahead, jump-back).
Arguments:
state::UInt64
: State of the RNG.δ
: Integer number representing number of the steps
to perform on the state
.
Nerf.count_and_generate_faces!
— Methodfunction count_and_generate_faces!(
indices, counters, @Const(vertex_grid), @Const(triangle_table),
@Const(log_densities), threshold::Float32)
Given log_densities
count number of indices that define triangles and optionally compute actual indices.
Arguments:
indices
: Output array of triangle indices. Passnothing
to only count number of indices, without generating them.counters
: Array of at least size2
. Number of indices that will be generated is written in the second element of it. Must be initialized to0
.vertex_grid
: 4D vertex grid computed bycount_and_generate_vertices!
that contains vertex ids.log_densities
: 3D grid of log-space densities that is used to determine face boundaries.threshold::Float32
: Log-space density threshold that defines boundary crossing.
Nerf.count_and_generate_vertices!
— MethodGiven log_densities
count number of vertices and optionally compute vertex coordinates contained in the render_bbox
bounding box.
How it works:
Given idx = (x, y, z)
coordinate of a cell in a grid of resolution
size, shift it by 1
along each dimension and test if there is density boundary-crossing between idx
and shifted coordinate idxₛ
.
Boundary crossing is when (log_densities[idx] > threshold) ≠ (log_densities[idxₛ] > threshold)
. If there is boundary crossing, then add new vertex and its id.
Nerf.diag
— Methoddiag(b::BBox)
Compute diagonal of the bbox, i.e. the width along each axis.
Nerf.enlarge
— MethodExpand bbox to enclose a point.
Nerf.expand_bits
— Methodexpand_bits(v::UInt32)
Expand 10-bit integer into 30 bits by inserting 2 zeros after each bit.
Nerf.generate_grid_samples_uniform!
— MethodGenerate samples on the grid bounded by render_bbox
, but in CS of train_bbox
, because they are then evaluated using trained model. Note that render_bbox
⊆ train_bbox
.
Nerf.get_marching_cubes_resolution
— MethodGiven mesh resolution and rendering bounding box in which it lies, compute marching cubes resolution.
Nerf.inflate
— Methodinflate(b::BBox, amount)
Expand bbox in both directions by the amount
.
Nerf.marching_cubes
— Methodmarching_cubes(
log_densities, threshold::Float32, render_bbox::BBox,
mc_resolution::SVector{3, UInt32})
Given 3D grid of log-densities perform marching cubes on it.
For more details, see count_and_generate_vertices!
and count_and_generate_faces!
methods.
Arguments:
log_densities
: 3D grid of log densities (non-exponentiated, i.e. log-space) which will be used to perform marching cubes. It must be ofmc_resolution
shape.threshold::Float32
: Log-space density threshold that determines boundary-crossing. If for a given point in the grid, log density is higher than thisthreshold
, it is considered to be inside of a mesh.render_bbox::BBox
: Bounding box that is used for rendering. Marching cubes is done in its domain.mc_resolution::SVector{3, UInt32}
: Resolution of the marching cubes sampling grid. Can be computed withget_marching_cubes_resolution
.
Returns:
- Vertex grid - 4D grid of
(3, mc_resolution...)
shape containing vertex ids. - Vertices - Vector of
SVector{3, Float32}
that contains vertex coordinates. - Indices - Vector of
UInt32
that contains triangle ids. Number of triangles is thenlength(indices) ÷ 3
.
Examples:
julia> mesh_resolution = 16;
julia> render_bbox = BBox(SVector{3, Float32}(0, 0, 0), SVector{3, Float32}(1, 1, 1));
julia> mc_resolution = INGP.get_marching_cubes_resolution(mesh_resolution, render_bbox);
julia> fake_densities = zeros(CPU(), Float32, tuple(Int64.(mc_resolution)...));
julia> fake_densities[2:3, 2:3, 2] .= 1f0;
julia> vertex_grid, vertices, indices = INGP.marching_cubes(fake_densities, 0.5f0, render_bbox, mc_resolution);
julia> length(vertices), length(indices)
(16, 84)
Raises:
ErrorException
iflog_densities
is not a 3D grid.ErrorException
iflog_densities
is not ofmc_resolution
shape.
Nerf.marching_cubes
— Methodmarching_cubes(
trainer::NerfTraining, render_bbox::BBox;
threshold::Float32 = 2.5f0, mesh_resolution::Int = 256)
Compute polygonal surface representation of NeRF using Marching Cubes algorithm.
This is a convenience wrapper around marching_cubes
, get_marching_cubes_resolution
, save_mesh
, trace_mesh_vertex_colors!
methods for NerfTraining
.
Arguments:
threshold::Float32
: Threshold that determines boundary of the surface. This threshold is for the log-space densities obtained by evaluating the model on sampled positions. Default value is2.5f0
.mesh_resolution::Int
: Defines grid resolution for marching cubes. Must be in[16, 2048]
range. Default is256
.
Returns:
true
if successfully created mesh, false
otherwise. It may fail if there are less than 3
vertices of less than 1
triangle.
References:
Nerf.marching_tetrahedra
— Methodmarching_tetrahedra(vertices, field; indices)
Convert scalar field encoded on the tetrahedral grid to triangle mesh using marching tetrahedra algorithm.
Arguments:
vertices
: Vertices of the tetrahedral mesh of3xN
shape andFloat32
eltype.field
: Scalar field evaluated at each vertex position ofN
length andFloat32
eltype.indices
: Tetrahedral mesh topology (face indices) of4xK
shape andUInt32
eltype.
Returns:
Vertex coordinates and triangle ids of the produced triangular mesh. Triangle mesh vertices are in the same coordinate system as input vertices
and of 3xN
shape. Triangle ids are in 3xK
shape.
References:
Nerf.materialize_ray_bundle!
— MethodArguments:
bundle::RayBundle
: Ray bundle.translations:: <: AbstractVector{SVector{3, Float32}}
: Vector of translations from the dataset.
Nerf.mesh_1_ring
— Methodmesh_1_ring(vertices, indices)
Arguments:
vertices
: Vector ofSVector{3, Float32}
vertex coordinates.indices
: Vector ofUInt32
indices starting from0
.
Nerf.morton3D
— Methodmorton3D(x::UInt32, y::UInt32, z::UInt32)
Compute a 30-bit Morton code for the given 3D point located in the unit cube [0, 1]³
.
Nerf.next_float
— Methodnext_float(state::UInt64)
Generate uniformly distributed Float32
number in [0, 1)
range using PCG32 RNG.
Arguments:
state::UInt64
: State of the RNG.
Nerf.next_uint
— Methodnext_uint(state::UInt64)
Generate uniformly distributed UInt32
number using PCG32 RNG.
Arguments:
state::UInt64
: State of the RNG.
Nerf.photometric_loss!
— MethodArguments:
rgba: <: AbstractMatrix{Float32}
: Output of the model. Must be of(4, N)
shape, where each 4 components are RGBA (A for density).
Refer to RayBundle
for documentation on thread_indices
, image_indices
and span
arguments.
Refer to RaySamples
for documentation on deltas
argument.
Nerf.random_vec2f0
— Methodrandom_vec2f0(state::UInt64)
Generate random SVector{2, Float32}
using PCG32 RNG.
Arguments:
state::UInt64
: State of the RNG.
Returns:
Vector and the new RNG state.
Nerf.random_vec3f0
— Methodrandom_vec3f0(state::UInt64)
Generate random SVector{3, Float32}
using PCG32 RNG.
Arguments:
state::UInt64
: State of the RNG.
Returns:
Vector and the new RNG state.
Nerf.relative_position
— Methodrelative_position(b::BBox, p)
Compute relative position of the point w.r.t. bbox. If the point is inside the bbox, then the resulting value would be a 3D vector in [0, 1]³
.
Nerf.subdivide_tetrahedra
— Methodsubdivide_tetrahedra(vertices::V, field::F; indices) where {V, F}
Subdivide tetrahedra mesh. Each tetrahedron is subdivided into 8 smaller ones.
Arguments:
vertices
: Vertices of the tetrahedral mesh of3xN
shape andFloat32
eltype.field
: Optional. Passnothing
to subdivide onlyvertices
andindices
. Scalar field evaluated at each vertex position ofN
length andFloat32
eltype.indices
: Tetrahedral mesh topology (face indices) of4xK
shape andUInt32
eltype.
Returns:
If field
is nothing
, then only subdivided vertices
and indices
. Otherwise vertices
, field
and indices
in the same shape as the input.
References: