Pointclouds and PCL Types

Introduction Caesar._PCL

A wide ranging and well used point cloud library exists called PCL which is implemented in C++. To get access to many of those features and bridge the Caesar.jl suite of packages, the base PCL.PointCloud types have been implemented in Julia and reside under Caesar._PCL. The main types of interest:

  • Caesar._PCL.PointCloud
  • Caesar._PCL.PCLPointCloud2
  • Caesar._PCL.PointXYZ
  • Caesar._PCL.Header
  • Caesar._PCL.PointField
  • Caesar._PCL.FieldMapper

The PointCloud types use Colors.jl:

using Colors, Caesar
using StaticArrays

# one point
x,y,z,intens = 1f0,0,0,1
pt = Caesar._PCL.PointXYZ(;data=SA[x,y,z,intens])

# etc.
struct PointCloud{T<:Caesar._PCL.PointT, P, R}

Convert a PCLPointCloud2 binary data blob into a Caesar._PCL.PointCloud{T} object using a field_map::Caesar._PCL.MsgFieldMap.

Use PointCloud(::Caesar._PCL.PCLPointCloud2) directly or create you own MsgFieldMap:

field_map = Caesar._PCL.createMapping(msg.fields, field_map)


  • Tested on Radar data with height z=constant for all points – i.e. 2D sweeping scan where .height=1.


  • TODO .PCLPointCloud2 convert not tested on regular 3D data from structured light or lidar yet, but current implementation should be close (or already working).


  • https://pointclouds.org/documentation/classpcl11pointcloud.html
  • (seems older) https://docs.ros.org/en/hydro/api/pcl/html/conversions8hsource.html#l00123

Conversion with ROS.PointCloud2

Strong integration between PCL and ROS predominantly through the message types

  • @rosimport std_msgs.msg: Header, @rosimport sensor_msgs.msg: PointField, @rosimport sensor_msgs.msg: PointCloud2.

These have been integrated through conversions to equivalent Julian types already listed above. ROS conversions requires RobotOS.jl be loaded, see page on using ROS Direct.

struct PointXYZ{C<:Colorant, T<:Number} <: Caesar._PCL.PointT

Immutable PointXYZ with color information. E.g. PointXYZ{RGB}, PointXYZ{Gray}, etc.


  • PointXYZRGB
  • PointXYZRGBA


  • https://pointclouds.org/documentation/structpcl11pointxyz.html
  • https://pointclouds.org/documentation/point__types8hppsource.html
struct Header

Immutable Header.

See https://pointclouds.org/documentation/structpcl11pclheader.html

struct PointField

How a point is stored in memory.

  • https://pointclouds.org/documentation/structpcl11pclpoint_field.html
struct FieldMapper{T<:Caesar._PCL.PointT}

Which field values to store and how to map them to values during serialization.

  • https://docs.ros.org/en/hydro/api/pcl/html/conversions8hsource.html#l00091
struct PCLPointCloud2

Immutable point cloud type. Immutable for performance, computations are more frequent and intensive than anticipated frequency of constructing new clouds.


  • https://pointclouds.org/documentation/structpcl11pclpoint_cloud2.html
  • https://pointclouds.org/documentation/classpcl11pointcloud.html
  • https://pointclouds.org/documentation/common2include2pcl2point__cloud8h_source.html

See also: Caesar._PCL.toROSPointCloud2

Aligning Point Clouds

Caesar.jl is currently growing support for two related point cloud alignment methods, namely:

ScatterAlign for Pose2 and Pose3

These factors use minimum mean distance embeddings to cost the alignment between pointclouds and supports various other interesting function alignment cases. These functions require Images.jl, see page Using Images.

ScatterAlign{P,H1,H2} where  {H1 <: Union{<:ManifoldKernelDensity, <:HeatmapGridDensity}, 
                              H2 <: Union{<:ManifoldKernelDensity, <:HeatmapGridDensity}}

Alignment factor between point cloud populations, using either

  • a continuous density function cost: ApproxManifoldProducts.mmd, or
  • a conventional iterative closest point (ICP) algorithm (when .sample_count < 0).

This factor can support very large density clouds, with sample_count subsampling for individual alignments.

Keyword Options:

  • sample_count::Int = 100, number of subsamples to use during each alignment in getSample.
    • Values greater than 0 use MMD alignment, while values less than 0 use ICP alignment.
  • bw::Real, the bandwidth to use for mmd distance
  • rescale::Real
  • N::Int
  • cvt::Function, convert function for image when using HeatmapGridDensity.
  • useStashing::Bool = false, to switch serialization strategy to using Stashing.
  • dataEntry_cloud1::AbstractString = "", blob identifier used with stashing.
  • dataEntry_cloud2::AbstractString = "", blob identifier used with stashing.
  • dataStoreHint::AbstractString = ""


arp2 = ScatterAlignPose2(img1, img2, 2) # e.g. 2 meters/pixel 


  • Supports two belief "clouds" as either
  • Stanard cvt argument is lambda function to convert incoming images to user convention of image axes,
    • Geography map default cvt flips image rows so that Pose2 +xy-axes corresponds to img[-x,+y]
      • i.e. rows down is "North" and columns across from top left corner is "East".
  • Use rescale to resize the incoming images for lower resolution (faster) correlations
  • Both images passed to the construct must have the same type some matrix of type T.
  • Experimental support for Stashing based serialization.


  • TODO Upgrade to use other information during alignment process, e.g. point normals for Pose3.

See also: ScatterAlignPose2, ScatterAlignPose3, overlayScanMatcher, Caesar._PCL.alignICP_Simple.


Future work may include ScatterAlignPose2z, please open issues at Caesar.jl if this is of interest.

Iterative Closest Point

Ongoing work is integrating ICP into a factor similar to ScatterAlign.


Align two point clouds using ICP (with normals).


using Downloads, DelimitedFiles
using Colors, Caesar

# get some test data (~50mb download)
lidar1_url = "https://github.com/JuliaRobotics/CaesarTestData.jl/raw/main/data/lidar/simpleICP/terrestrial_lidar1.xyz"
lidar2_url = "https://github.com/JuliaRobotics/CaesarTestData.jl/raw/main/data/lidar/simpleICP/terrestrial_lidar2.xyz"
io1 = PipeBuffer()
io2 = PipeBuffer()
Downloads.download(lidar1_url, io1)
Downloads.download(lidar2_url, io2)

X_fix = readdlm(io1)
X_mov = readdlm(io2)

H, HX_mov, stat = Caesar._PCL.alignICP_Simple(X_fix, X_mov; verbose=true)


  • Mostly consolidated with Caesar._PCL types.
  • Internally uses Caesar._PCL._ICP_PointCloud which was created to help facilite consolidation of code:
    • Modified from www.github.com/pglira/simpleICP (July 2022).
  • See here for a brief example on Visualizing Point Clouds.


  • TODO switch rigid transfrom to Caesar._PCL.apply along with performance considerations, instead of current transform!.

See also: PointCloud

Visualizing Point Clouds

See work in progress on alng with example code on the page 3D Visualization.