- Liu X, Yan M, Bohg J. Meteornet: Deep learning on dynamic 3d point cloud sequences[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 9246-9255.
- Carnegie Mellon University
Meteor Module
聚类方法
Direct grouping：主要灵感源于 时间过得越久，物体能移动的最大距离就越远。 因此对于邻域半径，也可以随着时间增加而增加。
\[ \mathcal{N}_{d}\left(p_{i}^{(t)} ; r\right)=\left\{p_{j}^{\left(t^{\prime}\right)} \mid\left\|\mathbf{x}_{j}^{\left(t^{\prime}\right)}-\mathbf{x}_{i}^{(t)}\right\|<r\left(\left|t^{\prime}-t\right|\right)\right\} \]
Chained-flow grouping：物体的运动通常沿着一定的轨迹方向。因此运动可以通过场景流[28]来进行描述。首先对于所有的时间\(t\)，利用已知的两个相邻帧的点云\(p_i^{t},p_j^{t-1}\)，通过场景流估计算子\(\mathcal{F}_{0}\)(如FlowNet3D[14])来找到\(p_i^{t}\)在\(t-1\)帧的对应虚拟点云\(p_i^{\prime(t-1)}\)，
\[ \left\{\mathfrak{f}_{i}^{(t, t-1)}\right\}_{i}=\mathcal{F}_{0}\left(\left\{p_{i}^{(t)}\right\},\left\{p_{j}^{(t-1)}\right\}\right) \]
估计得到的点云即 \(\mathbf{x}_{i}^{\prime(t-1)}=\mathbf{x}_{i}^{(t)}+\mathfrak{f}_{i}^{(t, t-1)}\)
为了找到\(p_i^{t}\)在\(t-2\)帧的对应虚拟点云，首先用上述同样的方式计算\(p_j^{t-1}\)在\(t-2\)帧的反向流\(\mathfrak{f}_{j}^{(t-1, t-2)}\)，然后通过\(p_i^{\prime(t-1)}\)和\(p_j^{t-1}\)的距离来对\(\mathfrak{f}_{j}^{(t-1, t-2)}\)进行一个加权插值得到\(\mathfrak{f}_{i}^{(t-1, t-2)}\)：
\[ \mathfrak{f}_{i}^{\prime(t-1, t-2)}=\frac{\sum_{j=1}^{k} w\left(\mathbf{x}_{j}^{(t-1)}, \mathbf{x}_{i}^{(t-1)}\right) \mathfrak{f}_{j}^{(t-1, t-2)}}{\sum_{j=1}^{k} w\left(\mathbf{x}_{j}^{(t-1)}, \mathbf{x}_{i}^{\prime(t-1)}\right)} \]
其中\(w\left(\mathbf{x}_{1}, \mathbf{x}_{2}\right)=\frac{1}{d\left(\mathbf{x}_{1}, \mathbf{x}_{2}\right)^{p}}\)为反距离函数，这样即可以得到\(t-2\)时刻的估计点云：\(\mathbf{x}_{i}^{\prime(t-2)}=\mathbf{x}_{i}^{(t)}+\mathfrak{f}_{i}^{(t, t-1)}+{\mathfrak{f}}_{i}^{(t-1, t-2)}\)。
对于\(t-2\)往前的帧都可以重复上述 chained 插值方式进行处理得到。最终可以根据虚拟点云来确定邻域范围为：
\[ \mathcal{N}_{c}\left(p_{i}^{(t)} ; r\right)=\left\{p_{j}^{\left(t^{\prime}\right)} \mid\left\|\mathbf{x}_{j}^{\left(t^{\prime}\right)}-\mathbf{x}_{i}^{\prime\left(t^{\prime}\right)}\right\|<r\right\} \]
参考文献
[28] Sundar Vedula, Simon Baker, Peter Rander, Robert Collins, and Takeo Kanade. Three-dimensional scene flow. In ICCV,1999
[14] Xingyu Liu, Charles. R. Qi, and Leonidas J. Guibas. Flownet3d: Learning scene flow in 3d point clouds. In CVPR, 2019.