IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 34, NO.

10, OCTOBER 2024 9633

3D Point Cloud Attribute Compression Using

Diffusion-Based Texture-Aware
Intra Prediction
Yiting Shao , Xiaodong Yang , Wei Gao , Senior Member, IEEE, Shan Liu , Fellow, IEEE,
and Ge Li , Member, IEEE
Abstract— There is an urgent need from various multimedia I. I NTRODUCTION
applications to efficiently compress point clouds. The Moving
Picture Experts Group has released a standard platform called
geometry-based point cloud compression (G-PCC). However,
its k-nearest neighbor (k-NN) based attribute prediction has
W ITH the significant progress of three-dimensional (3D)
sensing and acquisition technologies, a point cloud has
become a popular media format of 3D objects and scenes
limited efficiency for point clouds with rich texture and direc- in a variety of important multimedia applications such as
tional information. To overcome this problem, we propose a smart city, immersive communication, culture heritage storage,
texture-aware attribute predictive coding framework in a point
cloud diffusion model. In our work, attribute intra prediction is etc [1], [2], [3], [4], and [5]. Along with the rising market
solved as a diffusion-based interpolation problem, and a general requirements of 3D high-resolution representation, the data
attribute predictor is developed. It is theoretically proven that volume of point clouds is drastically increasing with abundant
G-PCC k-NN based predictor is a degraded case of the proposed geometric information and rich attributes. This situation brings
diffusion-based solution. First, a point cloud is represented heavy pressure on data transmission and storage systems with
as two levels of details with seeds as the inpainting mask
and non-seed points to be predicted. Second, we design point limited bandwidth and storage capacity. Compressing point
cloud partial difference operators to perform energy-minimizing clouds while maintaining high fidelity is an urgent demand
attribute inpainting from seeds to unknowns. Smooth attribute that has attracted significant attention from both industry and
interpolation can be achieved via an iterative diffusion process, academia. Notably, substantial progress has been achieved in
and an adaptive early termination is proposed to reduce complex- point cloud compression, particularly in the realms of geom-
ity. Third, we propose a structure-adaptive attribute predictive
coding scheme, where edge-enhancing anisotropic diffusion is etry compression [6], [7], [8] and attribute compression [9],
employed to perform texture-aware attribute prediction. Finally, [10], [11], [12], [13]. In this paper, we focus on point cloud
attributes of seeds are beforehand encoded and prediction resid- attribute compression.
uals of left points are progressively encoded into bitstream. The objective of point cloud attribute compression is to
Experiments show the proposed scheme surpasses the state-of- utilize attribute similarities to reduce the number of bits
the-art by an average of 14.14%, 17.52%, and 17.87% BD-BR
gains on the coding of Y, U, and V components, respectively. while keeping the quality of the decoded point cloud as
Subjective results on attribute reconstruction quality also verify close to the original one as possible. However, the dis-
the advantage of our scheme. crete and irregular geometric distribution of point clouds
Index Terms— Point cloud attribute compression, progressive makes it a challenging task to capture attribute correlation
intra prediction, diffusion-based interpolation, predictive coding. in 3D space. A typical work on point cloud attribute com-
pression is the Geometry-based Point Cloud Compression
(G-PCC) [14], a standard platform released by the Moving
Picture Experts Group (MPEG). G-PCC is an integrated point
cloud coding solution, including a set of geometry codecs
Manuscript received 4 December 2023; revised 30 March 2024; accepted (octree-based, predictive tree-based, and trisoup-based com-
28 April 2024. Date of publication 3 May 2024; date of current version pression schemes) and several attribute coding tools (region-
30 October 2024. This work was supported in part by the Natural Science
Foundation of China under Grant 62172021, Grant 62271013, and Grant adaptive hierarchical transform, attribute prediction and lifting
62031013; in part by the Guangdong Province Pearl River Talent Program transform).
under Grant 2021QN020708; and in part by the Shenzhen Science and Specifically, the attribute predictive coding in G-PCC is per-
Technology Program under Grant JCYJ20230807120808017. This article
was recommended by Associate Editor C. Herglotz. (Corresponding author: formed with three steps [14]: point cloud levels of detail (LoD)
Wei Gao.) generation via resampling, attribute intra predictor selection by
Yiting Shao and Wei Gao are with the School of Electronic and Computer the k-nearest neighbor (k-NN) search, and attribute prediction
Engineering (SECE), Shenzhen Graduate School, Peking University, Shenzhen
518055, China, and also with the Peng Cheng Laboratory, Shenzhen 518066, with the linear interpolation. However, several issues on the
China (e-mail: [email protected]; [email protected]). current G-PCC need to be considered. The first issue to
Xiaodong Yang and Ge Li are with the SECE, Shenzhen Graduate be addressed is the LoD generation. It only adjusts the
School, Peking University, Shenzhen 518055, China (e-mail: 2101212794@
stu.pku.edu.cn; [email protected]). sampling distance among points for different LoDs, which
Shan Liu is with the Media Lab, Tencent, Palo Alto, CA 94306 USA fails to ensure the regular geometric distribution of points
(e-mail: [email protected]). in each LoD. It is noted that points within lower LoDs are
Color versions of one or more figures in this article are available at
https://doi.org/10.1109/TCSVT.2024.3396694. more predominant as they are more frequently exploited for
Digital Object Identifier 10.1109/TCSVT.2024.3396694 attribute prediction. However, the current G-PCC does not pay
1051-8215 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
9634 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 34, NO. 10, OCTOBER 2024

Fig. 1. Pipeline of the proposed PDE-based progressive attribute prediction framework. In the first stage, a point cloud is decomposed into two LoDs with
seeds and unknown points. Seed attributes are coded into the bitstream and reconstructed as the attribute inpainting mask for the unknown points. In the
second stage, attributes of unknown points are iteratively interpolated by our PDE-driven inpainting module, and an adaptive early termination is introduced
to reduce complexity in this iterative process. In the third stage, using the interpolated point cloud as the predictor, texture-aware attribute predictive coding
is point-by-point performed after EED-based attribute diffusivity analysis. Finally, attribute predicted residuals of points are coded into the total bitstream.

more attention to the generation of lower LoDs. The second tive interpolation strategy with adaptive early termination
issue to be concerned is the predictor selection. The k-NN guarantees the effectiveness and efficiency of our scheme.
based selection adopts the Euclidean distance as the only • We design PDE operators with 3D inpainting masks for
metric, ignoring the geometric distribution of those selected point cloud attribute compression. By formulating the dis-
predictors. It may lead to a set of directional-biased predic- crete attribute prediction as a continuous diffusion-based
tors, which would impair attribute coding performance. The interpolation task, the proposed PDE-driven attribute pre-
third issue to be studied is the interpolation-based prediction. dictor provides a general solution to fully exploit attribute
The inverse distance weighted interpolation is a suboptimal spatial correlation. The current k-NN predictor is proved
way to generate predicted attributes with selected predic- to be a degraded solution of ours.
tors. A more sophisticated interpolation method needs to be • We construct a structure-adaptive attribute prediction
designed to fully utilize the attribute correlation in neighboring scheme with edge-enhancing anisotropic diffusion (EED),
regions. which can perform smooth prediction in local regions
There are some works [15], [16], [17] aimed at handling while edge details can be preserved. Texture-aware dif-
the aforementioned problems, but they only partially solve fusion tensors can be effectively extracted from the
those issues and lack the consideration of attribute predic- underlying texture structure to boost attribute prediction.
tion from the perspective of continuous attribute distribution. The rest of this paper is organized as follows. In Section II,
Wei et al. [15] provide a content-adaptive LoD generation we briefly review the works on point cloud attribute com-
method to reduce attribute prediction error, but they fail pression, diffusion-based image and point cloud processing.
to optimize the attribute predictor selection with the LoD Section III gives technical background, problem formulation,
structure. Chen et al. [16] improve the attribute predictor and an overview of the proposed scheme. Details about the
selection with a Hilbert space-filling curve based point order, proposed attribute prediction framework are elaborated in
but they ignore the impact of suboptimal attribute interpo- Section IV. Section V presents the experimental results and
lation step on coding performance. Song et al. [17] replace analysis. Finally, the paper is concluded in Section VI.
the interpolation-based attribute prediction in G-PCC with a
region-aware optimized graph transform, which improves the II. R ELATED W ORK
coding performance but brings high computational complexity. A. Point Cloud Attribute Compression
To overcome the aforementioned problems, we reformulate
Before conducting attribute compression, discrete and
the discrete attribute intra prediction task as a continuous
irregular point clouds are usually reorganized into com-
data interpolation problem in a diffusion process. Inspired
pact structures for better utilizing the local correlation of
by diffusion-based image codecs [18], [19], [20], we solve
attributes. Octree is the most popular representation of point
the problem with a partial differential equation (PDE)-driven
clouds. Schnabel and Klein [21] first adopt the octree in a
attribute prediction solution.
prediction-based point cloud codec. Later, Huang et al. [22]
As shown in Fig. 1, we propose a diffusion-based intra
propose an octree-based progressive attribute coding scheme
prediction framework for point cloud attribute compression.
to remove attribute redundancy in octree nodes. Different
Our main contributions are outlined as follows:
from octree-based codecs, Zhang et al. [23] first decom-
• We propose a progressive diffusion-based attribute pre- pose a point cloud into several graphs and adopt graph
dictive coding framework, which can perform accurate Fourier transform (GFT) to compress point cloud attributes.
texture-aware attribute prediction to improve attribute Cohen et al. [24] extend the work with an improved graph
coding performance. The proposed diffusion-based itera- construction method for more efficient attribute coding.
Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
SHAO et al.: 3D POINT CLOUD ATTRIBUTE COMPRESSION 9635

Then, Shao et al. [13] adopt the k-dimensional tree to conduct B. Diffusion-Based Image Processing
block partition before the graph construction. Shao et al. Diffusion models have shown excellent performance on
further provide a Lagrangian-optimized GFT in a block-based filling in missing data in image generation [29], [30], [31] and
attribute coding scheme [25]. Xu et al. [26] construct a inpainting [32], [33], [34]. A pioneer diffusion-based image
spatio-temporal GFT for dynamic point cloud attribute com- inpainting work is [34], where the image Laplacian serves
pression. Moreover, Song et al. [17] design a block-adaptive as a local smoothness operator and an anisotropic diffusion
graph transform method to improve attribute coding perfor- model is used to simultaneously inpaint missing regions along
mance. Considering GFT-based codecs are time-consuming, the direction perpendicular to the image gradient. Extend-
Queiroz and Chou [27] devise a more efficient transform-based ing this to image compression, many diffusion-based image
attribute coding scheme named RAHT, and this work is codecs [18], [19], [20] are designed to store a fraction of
adopted as a part of the solution in MPEG G-PCC. Recently, selected pixels and propagate textures to local regions by
G-PCC [14] has integrated a set of coding tools for efficient inpainting with PDEs. Different PDE operators would lead
attribute compression, while the way to exploit attribute spatial to different coding performances for different image types.
correlation still can be improved in G-PCC for better coding Mainberger et al. [35] propose a PDE-based codec with homo-
performance. geneous diffusion for cartoon-like image compression, but it
We highlight a selection of representative studies closely fails to handle images with complex textures. To overcome
related to our research and elaborate on the distinctive points this, Galić et al. [18] propose an image codec with EED-based
of our study compared to these works. Song et al. [17] interpolation, first showing that diffusion-based codecs with
introduce a block-adaptive codec for point cloud attribute suitable PDEs can outperform JPEG at low to medium bit
compression, incorporating progressive clustering and opti- rates. Schmaltz et al. [36] propose a new rectangular subdivi-
mized graph transform. In contrast, our work emphasizes sion and an enhanced entropy coder to improve the codec [18],
leveraging spatial attribute correlation to enhance intra pre- achieving better coding performance over JPEG 2000. Based
diction methods, leading to more compact spatial domain on this, Peter et al. [20] propose a uma-guided anisotropic
representations. Notably, our novel spatial diffusion model diffusion-based colorization scheme to achieve more efficient
facilitates redundancy removal, making our diffusion-based color compression. Those works prove that diffusion-based
intra prediction scheme a distinct feature. Zhao et al. [9] schemes have much potential in image compression tasks.
propose an image synthesis-based codec, mapping point clouds Drawing on the progress of diffusion-based techniques in
to 2D space for attribute image generation. In contrast, our tasks such as filling in missing data in image generation
approach preserves the 3D structure and exploits attribute and inpainting, we aim to leverage similar principles in our
correlation in 3D space. We introduce a spatial diffusion diffusion-based point cloud attribute compression.
model, enabling attribute intra prediction in 3D space, mark-
ing a significant distinction from Zhao’s projection-based
method. Liu et al. [10] present a hybrid codec integrating C. Diffusion-Based Point Cloud Processing
block-based intra prediction and sparse representation-based Similar to diffusion-based image processing, the basic idea
transform coding. In contrast, our two-layer point-wise intra of diffusion-based point cloud processing is to propagate from
prediction scheme fully exploits attribute spatial correlation known points to unknown points of local regions with smooth-
at the point level, with our texture-aware intra prediction ness priors and PDEs. However, extending PDEs from 2D
with an edge-enhancing diffusion model setting us apart images to 3D point clouds is a difficult problem because of the
from Liu’s approach. Xu et al. [26] propose a predictive sparse and discrete geometry structure of point clouds. A 3D
graph transform scheme for point cloud attribute compression, cuboidal space subdivision is proposed in a diffusion-based
focusing on spatial-temporal attribute correlation in dynamic 3D image codec [36], but it is not applicable for point cloud
point clouds. Unlike their emphasis on improved transform contents. Liang et al. [37] reconstruct the local surface of point
coding tools, we concentrate on enhancing intra-prediction clouds to derive differential operators in 3D space. Similarly,
methods to leverage spatial attribute correlation for static point Lai et al. [38] propose to construct local meshes and use
cloud compression. Our spatial diffusion model enables spatial numerical methods to approximate differential operators for
redundancy removal, distinguishing our scheme from Xu’s point clouds. Inspired by the nonlocal discrete regularization
work. Shen et al. [28] introduce a block-based attribute codec on weighted graphs for PDE-based image processing [39],
enhanced with a hierarchical sparse transform coding scheme. Lozes et al. [40] solve PDEs on graphs of point clouds with-
While their emphasis is on hierarchical transform coding, out the extra construction of locally approximated surfaces.
we focus on enhancing intra prediction methods to miti- However, the calculation of PDEs on point cloud graphs is
gate spatial redundancy. Our texture-aware attribute predictive time-consuming. Hooda and Pan [41] adopt the 3D Sobel
coding framework, within a point cloud diffusion model, operator to obtain the point cloud luma gradient by assuming
stands out as a distinctive feature compared to Shen’s work. the current block with the associated 18 neighboring blocks
Introducing a novel diffusion-based intra prediction scheme are fully occupied. Nevertheless, this assumption would not
for point cloud attribute compression, particularly through hold in point clouds with an irregular geometry distribution.
the integration of spatial diffusion models and texture-aware Designing a suitable PDE operator to effectively reconstruct
predictive coding frameworks, distinguishes our work from missing data from the provided points presents a significant
existing literature. challenge. Besides, the design of an efficient predictive coding

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
9636 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 34, NO. 10, OCTOBER 2024

method for the inpainted points is important for a PDE-based Let w(u, v) be the weight of the edge (u, v) connecting
point cloud attribute codec. In this paper, we focus on these vertices u ∈ V and v ∈ V . Given a function f : V →
key problems. R3 mapping each vertex u to its attribute value f (u), the
difference operator of f on edge (u, v) ∈ E is computed by:
III. BACKGROUND , P ROBLEM F ORMULATION , AND p
OVERVIEW OF O UR PDE-BASED S OLUTION (d f )(u, v) = w(u, v)( f (v) − f (u)). (4)
A. Background The directional derivative of f at vertex u along the edge
1) G-PCC Interpolation-Based Prediction: In G-PCC, the e = (u, v) is defined as:
attribute coding scheme performs an interpolation-based pre-
∂f
diction with reconstructed neighboring points [14]. More = ∂v f (u) = (d f )(u, v). (5)
precisely, when compressing the attribute of point i, the set ∂e u
of decoded k-nearest neighbors Ni with the reconstructed The weighted gradient operator of function f at vertex u is
attributes f 0 ( j) j∈Ni are found via the Euclidean distance met- defined as a vector with a set of directional derivatives along
ric. Then, the predicted attribute f (i) of point i is calculated the edge (u, v) between vertex v and u as follows:
by an inverse distance weighted interpolation as follows:
∇w f (u) = (∂v f (u) : (u, v))⊤ ,
j∈Ni di j 2 f 0 ( j)
1
 P 
1
⊤
f (i) = Round · P 1
, (1) = ∂v1 f (u), . . . , ∂vk f (u) , ∀ (u, vi ) ∈ E. (6)
k j∈N 2
i di j
By leveraging the connectivity information embedded in
where Round(·) represents a rounding operation on a weighted graphs, graph-based PDEs demonstrate superior per-
floating-point number to the nearest integer. di j denotes the formance compared to traditional PDE-based techniques [37],
distance between the point i and its neighboring point j. [38], [39]. However, two main weaknesses in the graph-based
k means the number of neighbors. PDE method warrant consideration. First, the complexity of
2) Continuous Diffusion-Based Inpainting: Consider a con- graph construction from point clouds poses computational
tinuous 3D data domain u(x) :  → R3 maps 3D space challenges due to intricate edge creation and weight assign-
to associated values [42]. With the known inpainting mask ment processes. Second, the construction of graph-based PDEs
J ⊂ , a classic diffusion-based inpainting process usually relies on various parameters, such as the p-value in the
derives a reconstruction f (x) :  → R3 for the missing data p-Laplacian operator, weight function, and patch size, across
x ∈  \ J by solving the following differential equation on diverse data types, significantly influencing results. Yet, there
the inpainting mask J : are no clear guidelines for optimal parameter selection. Hence,
(1 − c(x))L f (x) − c(x)( f (x) − u(x)) = 0, (2) there is a need to investigate a more robust and efficient PDE
operator for point clouds.
where L is the difference operator, and c(x) is the character-
istic function of x that is 1 when x ⊂ J , and 0 elsewhere. The
first term introduces a smoothness constraint on the missing B. Problem Formulation
data via the difference operator, while the second term ensures By formulating the discrete attribute prediction as a 3D data
that the values are preserved for the inpainting mask J . continuous interpolation with a diffusion model, we extend the
The continuous solution of the diffusion-based inpainting distance-based attribute interpolation in G-PCC to a general
process in Eq. (2) can be obtained by minimizing the diffusion diffusion-based inpainting solution. The principle is that only a
energy defined by the following equation: subset of specifically chosen points is coded and stored, while
attributes of the remaining regions are reconstructed using the
Z
1
min E( f (x)) = ∥L f (x)∥2 d x, (3) PDE-based inpainting operator.
f (x) 2 
Let us consider a 3D point cloud domain  ⊂ R3 with
where ∥ · ∥ means the Euclidean norm.
a set of decoded points J ⊂  as the inpainting mask.
3) Point Cloud Difference Operator Construction: A solu-
Let f 0 ( j) describe the known attribute of the decoded point
tion of the PDE operator in 3D space is provided on point
j ∈ J . The attribute function f :  → R3 maps 3D positions
cloud graphs in Lozes’s work [40]. To tackle the challenges
to attribute values, which is initialized with f (i) = f 0 (i)
posed by traditional PDE approaches [37], [38], [39] when
when i ∈ J . Our goal of PDE-based attribute inpainting
handling irregular and sparse geometries within point clouds,
is to reconstruct the attributes f (i) of the unknown point
a graph-based framework is proposed to leverage the connec-
i ∈  with a set of known points J in the inpainting
tivity structure of weighted graphs to define PDE operators.
domain  \ J based on smoothness assumptions. To this aim,
The graph-based PDE operators enable accurate computa-
we embed the diffusion-based attribute propagation process in
tion of gradients and directional derivatives in point clouds.
the following PDE:
Designed to capture the intrinsic geometric properties of the
data, these PDEs effectively mitigate the effects of irregular (1 − c(i))L f (i) − c(i)( f (i) − f 0 (i)) = 0, (7)
and sparse distributions in point clouds. Let G = (V, E, w) be
a weighted graph of a point cloud, where V = {v1 , . . . , vk } is where L denotes the difference operator. Our PDE-based
a set of k vertices and E is a set of undirected weighted edges. attribute painting is given by the steady state of the evolution

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
SHAO et al.: 3D POINT CLOUD ATTRIBUTE COMPRESSION 9637

equation: To solve the problem in Eq. (13), we convert it to the

following formulation and make its derivative equal to zero:
∂ f = L f on  \ J, (8)
∂ E( f (i)) X
f = f 0 on J, (9) =2 w(i, j)( f (i) − f ( j)),
∂ f (i)
j∈Ni
where the choice of the difference operator L plays a crucial = 0. (14)
role in the attribute inpainting quality.
To achieve structure-adaptive attribute interpolation, Then, the predicted attribute f (i) of point i can be obtained
we introduce anisotropic diffusion in the construction of the by the following formulation:
difference operator with L f = div(D∇ f ), and the Eq. (8) is
j∈N w(i, j) f ( j)
P
presented as: f (i) = P i . (15)
j∈Ni w(i, j)
∂ f = div(D∇ f ), (10) It is noted that the derived solution in Eq. (15) is in the
where D is the diffusion tensor calculated by the attribute same formulation of the G-PCC interpolation-based predictor
information around the point to be inpainted. in Eq. (1), by setting w(i, j) as the reciprocal of the square
As presented in Eq. (3), the inpainting solution on continu- of the distance between point i and j. We prove that the
ous space x ∈  can be obtained by minimizing the diffusion G-PCC k-NN based attribute predictor is a degraded case of
energy. Therefore, we can obtain the PDE-based attribute our PDE-based attribute prediction solution.
inpainting solution by minimizing the discretization formu-
lation of the attribute diffusion energy defined as follows: D. Overview of Our PDE-Based Attribute Prediction Scheme
1X Based on the proposed diffusion-based attribute inpainting
min E( f (i)) = ∥D∇ f (i)∥2 , (11) formulation, we propose a texture-aware attribute predictive
f (i) 2
i∈ coding framework in a PDE-based diffusion model. Fig. 1
Finally, we formulate the attribute prediction problem in presents the pipeline of the proposed framework with three
Eq. (1) as a local gradient optimization problem defined as processing stages: Point cloud decomposition, PDE-driven
follows: attribute interpolation, and EED-based attribute predictive
coding. First, a point cloud is represented as two LoDs
min ∇ f (i)⊤ D ⊤ D∇ f (i), (12) with seeds and non-seed points via geometry distribution
f (i)
analysis. Seed attributes are coded into the bitstream and then
where the design of the PDE operator ∇ f (i) and the diffusion decoded as the attribute inpainting mask for non-seed points.
tensor D has a vital effect on the attribute reconstruction Then, attributes of non-seed points are iteratively generated
quality and the efficiency of the compression. We provide the by our PDE-driven attribute inpainting module. Moreover,
detailed implementation of the PDE-based iterative optimiza- we introduce an adaptive early termination to speed up the
tion process in Section IV. iterative process. Later, EED-based attribute predictive coding
is conducted point by point based on the interpolated point
cloud and local diffusion tensors of inpainted points. Finally,
C. Generalized Diffusion-Based Attribute Inpainting Solution attribute predicted residuals of points are coded into the
In Section III-B, we provide the diffusion-based attribute bitstream.
inpainting formulation in Eq. (7) and the PDE-based attribute
prediction solution in Eq. (11). In this part, we illustrate that IV. O UR A PPROACH
the G-PCC k-NN based attribute predictor in Eq. (1) is a A. Point Cloud Decomposition for LoD Generation
special case of our generalized diffusion-based solution.
We represent a point cloud as a two-layer LoD with seeds
The k-NN structure in G-PCC can be regarded as a graph
and non-seed points. As point cloud attribute prediction is
with the vertex i and its neighbors j ∈ Ni of the point cloud.
formulated as a general diffusion-based inpainting problem in
We define the PDE operator ∇ f (i) as the graph-based PDE
Section III-B, the goal of the inpainting process is to exploit a
operator in Eq. (6) and set the diffusion tensor D as the identity
set of seeds as the inpainting mask to reconstruct the attributes
matrix. To get the predicted
√ attribute f (i), we plug in the PDE
of the left points. To ensure effective attribute diffusion with
operator ∇w f (i) = w(i, j)( f (i) − f ( j)) and the diffusion
3D inpainting masks in Fig. 2, a seed generation method is
tensor D = I into the proposed attribute inpainting solution
proposed based on geometric distribution analysis. We assume
in Eq. (11), and we can get the following formulation:
X that the point cloud geometry is decoded and voxelized.
min E( f (i)) = min ∥D∇w f (i)∥2 , Details of our LoD generation process are as follows.
f (i) f (i) First, we arrange all 3D-space points into a 1D point
j∈Ni
X sequence with decoded geometry by the Morton order. Then,
= min (∇w f (i)⊤ D ⊤ D∇w f (i)),
f (i) the LoD generation is performed iteratively over all reordered
j∈Ni
X points. In the beginning, all points are marked as non-visited.
= min w(i, i)( f (i) − f ( j))2 . (13) Two LoDs L 0 and L 1 are constructed for gathering seeds
f (i)
j∈Ni and non-seed points, respectively. The process starts from

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
9638 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 34, NO. 10, OCTOBER 2024

Fig. 3. The proposed local region division template for calculating the inner
product of two directional derivatives in the PDE-based inpainting process.
We only select 8 neighbors with no displacement along the z-axis of point J .
To reduce complexity, nine discrete locations are clustered into four local
Fig. 2. The proposed 3D PDE-based inpainting mask with the cuboidal regions R0 , R1 , R2 , and R3 . Then, partial derivatives G x (J ), G y (J ), and
subdivision. To get the inpainted attributes f (x, y, z) for point J with the G z (J ) of point J are derived with associated points in the local region.
positions (x, y, z), 18 neighboring locations centered around point J are
selected for the calculation of partial derivatives G x (J ), G y (J ), and G z (J ). 18 neighboring locations. We define the attribute gradient at
point J with the 3D geometric position (x, y, z) as follows:
the first point, and we mark it as visited and assign it to
the seed set L 0 . Along the process, the current point would ∇ f (J ) = ∇ f (x, y, z) = (G x (J ), G y (J ), G z (J ))⊤ ,
∂ f (x, y, z) ∂ f (x, y, z) ∂ f (x, y, z) ⊤
 
be ignored without any actions when it has been visited.
= , , , (16)
Otherwise, we mark it as visited and add it to L 0 . Moreover, ∂x ∂y ∂z
local geometry analysis is performed for each seed, where all
where G x (J ), G y (J ), and G z (J ) denotes the partial deriva-
occupied neighbors sharing an edge or a face with the current
tives of the point J ’s attribute in three dimensions.
point would be evaluated: If they are non-visited, we mark
With the proposed point cloud cuboidal subdivision pre-
them as visited and add them into L 1 ; Otherwise, no actions
sented in Fig. 2, we define G x , G y , and G z as follows:
would be taken. The proposed geometry distribution analysis
ensures that every non-seed point in L 1 would have no less 1 1 X
G x (J ) = ∗ [( ∗ o(X )( f (X ) − f (J )))
than one corresponding seed in L 0 at the same 3D inpainting 10 2
X =D,H,N ,R
mask. 1 X
−( ∗ o(X )( f (X ) − f (J )))
2
B. Point Cloud PDE Operator Construction X =B,F,L ,P

It is difficult to directly extend 2D image PDEs [20] to 3D + o(K )( f (K ) − f (J )) − o(I )( f (I ) − f (J ))],

PDEs by replacing a 2D rectangular subdivision with a 3D (17)
cuboidal subdivision due to the sparsity in point cloud geom- 1 1 X
etry. To address the limitations associated with graph-based G y (J ) = ∗ [( ∗ o(X )( f (X ) − f (J )))
10 2
point cloud PDE operators [40], we develop a robust 3D X =A,F,H,O
PDE operator tailored to handle the uncertainty from sparse 1 X
−( ∗ o(X )( f (i) − f (J )))
geometry distributions within point clouds. Our goal is to 2
X =E,L ,N ,S
efficiently calculate gradients and directional derivatives for
+ o(G)( f (G) − f (J )) − o(M)( f (M) − f (J ))],
accurate prediction solutions in point cloud attribute com-
pression. In the proposed method, we introduce a novel 3D (18)
PDE-based inpainting mask featuring cuboidal subdivision. 1 1 X
G z (J ) = ∗ [( ∗ o(X )( f (X ) − f (J )))
This mask is utilized to compute the inpainted attributes for 10 2
X =A,B,D,E
each point within the point cloud.
1 X
As shown in Fig. 2, we select 18 neighboring locations to −( ∗ o(X )( f (X ) − f (J )))
compute the attribute gradient of point J , aiming to ensure 2
X =O,P,R,S
both gradient accuracy and computational simplicity. These + o(C)( f (C) − f (J )) − o(Q)( f (Q) − f (J ))],
neighboring locations are specifically selected to share an
(19)
edge or a face with the central point. This preserves the
necessary geometry correlation for accurate diffusion-based where X is the neighboring location of point J , and it can
attribute interpolation. The utilization of 18 neighbors aligns be 18 neighbors centered around J . o(X ) is the occupancy of
with the design of our PDE operator, as these neighbors location X . o(X ) is 1 when the location is occupied by points.
are easily identifiable with the cuboidal subdivision on Otherwise, o(X ) is 0. The scalar 1/10 means the average of ten
the point cloud. Moreover, the computational efficiency of PDE operators calculated by associated neighbors. The scalar
our approach benefits from this predetermined number of 1/2 is the distance-based weight of co-planar neighbors, where
neighbors. Notably, while the central point is occupied, its 2 is the Manhattan distance between the current point and its
neighboring regions may vary in occupancy status. Despite this co-planar neighbors.
variability, the proposed method ensures accurate calculation As shown in Fig. 3, when calculating the inner product
of attribute gradients at the central point by selecting these of two directional derivatives, we exploit the point J and

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
SHAO et al.: 3D POINT CLOUD ATTRIBUTE COMPRESSION 9639

Fig. 4. Pipeline of the proposed PDE-driven attribute iterative interpolation scheme with adaptive early termination. It performs energy-minimizing attribute
inpainting from seeds to non-seed points in an iterative update process.

its 8 neighboring points with no movement along the z-axis. overcoming their limitations and providing effective solutions
There are 28 = 512 possible occupancy situations of these for handling complexities in point cloud attribute compression.
8 neighbors. To reduce complexity, we divide nine discrete
points into four local regions R0 , R1 , R2 , and R3 , where C. PDE-Driven Attribute Iterative Interpolation With
each group Ri contains four locations (P0 , P1 , P2 , P3 ). For Adaptive Early Termination
example in the local region R0 , as presented in Fig. 3,
Fig. 4 presents the pipeline of the proposed PDE-driven
the (P0 , P1 , P2 , P3 ) represents the neighbors (F, G, I, J ).
attribute iterative interpolation scheme to perform energy-
To illustrate our definition of the inner product of two
minimizing attribute inpainting from seeds to non-seed points.
directional derivatives, we use the inner product derivatives
1) Energy-Minimizing Attribute Interpolation: With our
G x y (Ri ) of the x and y axes for the example. Therefore, G xi
problem formulation in Section III-B, we perform diffusion-
and G yi in the region Ri are derived by:
based attribute interpolation by minimizing the total attribute
1 energy of all seeds v ∈ Nseed and non-seed points u ∈ NnonSeed
G xi = ∗ [o(P1 ) ∗ o(P0 ) ∗ ( f (P1 ) − f (P0 )) in the point cloud as follows:
4
+ o(P3 ) ∗ o(P2 ) ∗ ( f (P3 ) − f (P2 )) X X
min wseed E( f (v)) + wnonSeed E( f (u)), (23)
1 v∈Nseed
+ ∗ o(P1 ) ∗ o(P2 ) ∗ ( f (P1 ) − f (P2 )) u∈NnonSeed
2
1 where wseed and wnonSeed are the attribute energy weights of
+ ∗ o(P3 ) ∗ o(P0 ) ∗ ( f (P3 ) − f (P0 ))], (20) seeds and non-seed points, respectively. As attributes of seeds
2
1 are more reliable than inpainted attributes of non-seed points,
G yi = ∗ [o(P0 ) ∗ o(P2 ) ∗ ( f (P0 ) − f (P2 )) we set wseed to be greater than wnonSeed .
4
The local attribute energy of a point with the position
+ o(P1 ) ∗ o(P3 ) ∗ ( f (P1 ) − f (P3 ))
(x, y, z) is defined as:
1
+ ∗ o(P1 ) ∗ o(P2 ) ∗ ( f (P1 ) − f (P2 )) E( f (x, y, z)) = ∇ f (x, y, z)⊤ T(x,y,z) ∇ f (x, y, z), (24)
2
1
+ ∗ o(P0 ) ∗ o(P3 ) ∗ ( f (P0 ) − f (P3 ))], (21) where the attribute gradient ∇ f (x, y, z) is a vector with partial
2 derivatives G x , G y , and G z along the x, y, and z axes,
where the scalar 1/4 means the average of four PDE operators representing the rate of attribute changes in 3D space. The
calculated in a 2 × 2 local area. The scalar 1/2 is the diffusion tensor T(x,y,z) is a 3 × 3 matrix that guides the
distance-based weight of co-planar neighbors, where 2 is the diffusion process by specifying the amount of diffusivity in
Manhattan distance between the current point and its co-planar the diffusion directions. In our implementation, we use the
neighbors. diffusion tensor T to present D ⊤ D in Eq. (12).
Then, the inner product derivatives G x y of x and y axes The choice of the diffusion tensor T(x,y,z) is impor-
around the point J is computed via averaging the sum of G xi ∗ tant for point cloud attribute inpainting quality. To enable
G yi in local regions: edge-enhancing anisotropic inpainting in the diffusion-based
attribute interpolation, we define T with the Charbonnier
3
1 X diffusivity function g [43] as follows:
Gxy = ∗ G xi G yi . (22)
4  
i=0 T = g ∇ f σ ∇ f σ⊤ , (25)
In contrast to graph-based PDEs [40], the proposed PDEs   1
operate directly on point clouds without the need for graph g ∇ f σ ∇ f σ⊤ := q , (26)
(∇ f σ ∇ f σ⊤ )
construction or manual parameter tuning. This approach 1+ λ2
w(i, j) f ( j)
P
streamlines computation and guarantees robust performance j∈N
across different types of point clouds. By harnessing intrin- f σ (i) = P i , (27)
j∈Ni w(i, j)
sic geometric properties inherent in point clouds, our PDE
w(i, j) = e−di, j /2σ ,
2 2
operator presents an efficient alternative to graph-based PDEs, (28)

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
9640 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 34, NO. 10, OCTOBER 2024

where f σ (i) is the Gaussian smoothing of the attribute f (i) on Algorithm 1 Proposed Attribute Iterative Interpolation With
point i by a Gaussian with standard deviation σ , and w(i, j) is Adaptive Early Termination Scheme
the weighted attribute correlation defined by the L2 distance Input: Initial attributes f 0 , seed LoD L 0 , non-seed LoD L 1 .
di, j between point i and j. λ is a contrast parameter. Ni are Output: Interpolated attributes f , diffusion tensor T .
neighbors being co-linear and co-planar to point i. 1: Claculate PDE operator ∇ f , initial T = I , and compute
To simplify the way to calculate the diffusion tensor T in Gaussian filtered attributes f σ .
Eq. (25), we decompose T as a symmetric matrix by: 2: for point u in L 0 do
3: flaginpaint (u) = False, flagupdate (u) = False.
g(∇ f σ ⊤ ∇ f σ ) 0 0
 
4: end for
T = (n σ , n ⊥ , n × )  0 1 0  (n σ , n ⊥ , n × )⊤
5: for point u in L 1 do
0 01
  6: flaginpaint (u) = True, flagupdate (u) = False.
100 7: end for
= (n σ , n ⊥ , n × )  0 1 0  (n σ , n ⊥ , n × )⊤ 8: for iteration i = 1 : N do
001 9: for point u in L 0 + L 1 do
1 − g(∇ f σ ⊤ ∇ f σ ) 0 0 if flaginpaint (u) = False then
 
10:
− (n σ , n ⊥ , n × )  0 0 0  (n σ , n ⊥ , n × )⊤ 11: continue
0 00 12: end if
13: Compute energy-minimizing inpainted attribute
= I + (g(∇ f σ ⊤ ∇ f σ ) − 1)n σ n ⊤
σ, (29)
f (u).
where I is a 3 × 3 identity matrix, and n σ is the normal vector 14: if f (u) = f 0 (u) then
of the gradient ∇ f σ . n ⊥ and n × are two orthogonal vectors 15: flaginpaint (u) = False.
of n σ . (σ, n ⊥ , n × ) forms an orthogonal matrix. 16: else
Plugged our designed PDE operator ∇ f (x, y, z) in Eq. (16) 17: f 0(u) = f (u), and flagupdate (u) = True.
and the diffusion tensor T in Eq. (29) into the energy function 18: Update flagupdate (v) = True for all neighbors v in
in Eq. (24), we obtain the final formulation of the attribute the point u’s inpainting mask.
energy function on the point (x, y, z) by: 19: end if
20: end for
E( f (x, y, z)) = ∇ f (x, y, z)⊤ T(x,y,z) ∇ f (x, y, z) 21: for u in L 0 + L 1 do
= T(x,y,z) (0, 0)G x 2 + T(x,y,z) (1, 1)G y 2 22: if flagupdate (u) = True then
+ T(x,y,z) (2, 2)G z 2 + 2 ∗ T(x,y,z) (0, 1)G x y 23: Update f σ (u).
+ 2 ∗ T(x,y,z) (0, 2)G x z 24: end if
25: end for
+ 2 ∗ T(x,y,z) (1, 2)G yz . (30) 26: for u in L 0 + L 1 do
By substituting Eq. (24)-(30) into Eq. (23), the total energy 27: if flagupdate (u) = True then
minimization problem in Eq. (23) can be reformulated as 28: Update T (u).
a quadratic optimization problem. We obtain the optimal 29: end if
inpainted attributes by minimizing the quadratic expression 30: end for
of point cloud attributes through an iterative process with 31: end for
adaptive early termination. 32: return f and T .
2) Attribute Iterative Inpainting Strategy With Adaptive
Early Termination: Instead of separately minimizing the local
energy for each point, we perform the energy-minimizing regions with complex textures usually need more iterations
attribute inpainting for all non-seed points simultaneously to achieve a good inpainting effect than those with relatively
for the best outcomes. This process ensures the PDE-driven smooth attributes. That is the reason why we introduce an
attribute inpainting can fully utilize the attribute spatial corre- adaptive early termination in the iterative attribute inpainting
lation of neighbors in a wide-range diffusion process of point process. If there are still points to be updated after one
clouds. round of inpainting, the iteration continues, and attribute PDE
As presented in Fig. 4, we input the attributes of seed and operators and diffusion tensors would be recalculated with
initial attributes of non-seed points in the pipeline. Attribute the latest-updated attributes of points. When the iterative
PDE operators and diffusion tensors are first calculated for process ends, we finally obtain the inpainted attributes for
all points. Interpolated attributes are derived by solving all non-seed points. Details of the proposed attribute iterative
the total energy minimization problem with PDE operators interpolation with an adaptive early termination scheme are
and diffusion tensors. After updating non-seed points with given in Algorithm 1.
interpolated attributes in an iteration, we introduce an early
termination decision module to evaluate the reconstruction
status of the current attribute interpolation. It is noted that, D. EED-Based Attribute Predictive Coding
when the complexity of local textures varies, the required Because attributes of seeds have already been coded into
iterations for a better attribute interpolation also vary. Local the bitstream, the attribute predictive coding is performed on

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
SHAO et al.: 3D POINT CLOUD ATTRIBUTE COMPRESSION 9641

non-seed points in a point-to-point manner. Considering there

exists the directional bias in point cloud interpolated attributes,
it would be desirable to guide the prediction towards the
orientation of underlying attribute distribution. This bias arises
from spatial variations or patterns within the distribution,
where certain attributes are more pronounced along specific
orientations within the point cloud. This phenomenon is partic-
ularly notable in regions characterized by intricate textures and
abundant edges. Therefore, we introduce the edge-enhancing
diffusion (EED) predictor for texture-aware attribute predic-
tion. Details of our EED-based attribute predictive coding
scheme are as follows.
We use the interpolated attributes generated from our
aforementioned PDE-driven inpainting as attribute predictors
of non-seed points to be coded. Then, attribute prediction
residuals are quantized and entropy-coded into the bitstream.
Meanwhile, those residuals are decoded and reconstructed to
update the interpolated attribute of the current non-seed. Later,
texture-aware diffusivity analysis is performed to capture the
change in the local attribute structure. To avoid the prediction
error accumulation and improve the predictive accuracy of
left non-seed points, we use the recalculated diffusion tensor
in an EED-based attribute inpainting process to generate a
new version of the interpolated attributes of left non-seed
points. We divide all points into three categories based on their
coding status: coded seeds u ∈ N0 , predicted non-seed points Fig. 5. Test point cloud datasets. From left to right and up to down: Andr ew,
v ∈ N1 , and non-seed points to be predicted i ∈ N2 . Then, Ricar do, David, Phil, Sarah, Facade, Queen, T haidancer , Soldier ,
Longdr ess, Redandblack, Loot, Basketball and Dancer .
the updated attributes f (i) of non-seed points to be predicted
i are derived by minimizing the total attribute energy function
defined as: V. E XPERIMENTAL R ESULTS
X X
min E( f (i)) = w0 E( f (u)) + w1 E( f (v)) A. Experimental Setup
f (i)
u∈N0 v∈N1 We conduct a series of experiments to validate the effec-
X
+ w2 E( f (i)), (31) tiveness of the proposed point cloud attribute prediction
i∈N2 framework. A set of standard point clouds with different
attribute characteristics from MPEG [44] and JPEG [45] are
where w0 , w1 , and w2 are attribute energy weights of seeds, tested for point cloud attribute compression. Test datasets are
predicted non-seed points, and non-seed points to be predicted, visualized in Fig. 5. We compare the proposed framework with
respectively. As points in N0 are deemed more reliable than several competitive platforms from industry and academia:
those in N1 , and similarly, points in N1 are considered more (i) MPEG G-PCC(v23), (ii) BAAC (Block-Adaptive Attribute
reliable than those in N2 , we assign weights accordingly. Coder [17]), and (iii) HAC (Hybrid Attribute Coder [25]).
Specifically, w0 is set to be greater than w1 , and w1 is set About the evaluation metrics, we follow the MPEG com-
to be greater than w2 . mon test conditions [44] and use the bits per point (bpp)
The solving process of our energy minimization problem to denote the total attribute bitrate. The Bjontegaard Delta
is roughly the same as the previous attribute interpolation Bitrate (BD-BR) value [46] is adopted to measure average
process in Section IV-C. The difference is that we directly bitrate differences between rate-distortion (R-D) curves for
use attributes f to compute the diffusion tensor in Eq. (32) the same objective quality. Besides, the Peak Signal-to-Noise
instead of using the Gaussian smoothing f σ . Ratio (PSNR) is calculated to measure the reconstructed
  attribute distortion. Once PSNRY , PSNRU , and PSNRV of
D = g ∇ f∇ f⊤ . (32) attribute components Y, U, and V are collected, the combined
PSNRYUV can be computed as PSNRYUV = (6PSNRY +
In this way, we can get the updated attribute predictors PSNRU + PSNRV )/8.
for non-seed points to be predicted. Our EED-based attribute All experiments are performed on a computer with an
prediction scheme can fully utilize the attribute spatial cor- Intel i7 8700K CPU (3.7GHz), and 64GB RAM. In the
relation for better texture-aware attribute prediction. Finally, attribute iterative interpolation of Section IV-C, we set wseed
attribute residuals of all non-seed points are encoded into the as 6 and wnonSeed as 1 in Eq. (23). The maximum of iterations
total bitstream. is 30. Regarding the diffusion tensor calculation in Eq. (25),

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
9642 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 34, NO. 10, OCTOBER 2024

TABLE I
P ERFORMANCE C OMPARISONS OF THE P ROPOSED F RAMEWORK W ITH HAC, BAAC, AND G-PCC ON P OINT C LOUD ATTRIBUTE L OSSY C OMPRESSION

the Gaussian standard deviation σ is set as 3, and the λ clouds with distorted attributes generated by our scheme and
of diffusivity is set as 0.6. In the attribute prediction of comparative platforms. Datasets Soldier and T haidancer
Section IV-D, we set w0 as 6, w1 as 4, and w2 as 1 in Eq. (31). with different attribute characteristics and rich textures are
For attribute lossy compression, the quantization step of seeds chosen in this quality assessment. Fig. 7 presents the subjective
qpseed is set as max(1, qpnonSeed /2), in which qpnonSeed is the quality comparison of decoded point clouds between our
quantization step of non-seed points. scheme and the state-of-the-art methods at similar attribute
bitrates. We zoom in point clouds in the red box for a better
B. Comparisons With the State-of-the-Art Methods representation. From the contents in the yellow circle, we see
that the proposed method can better preserve attribute details,
1) Compression Performance Evaluation: Table I sum-
especially in areas with complex textures and rich edges.
marizes the performance comparison of our scheme and
Moreover, the objective quality comparison between our
comparative platforms HAC, BAAC, and G-PCC(v23) on
scheme and the state-of-the-art methods is presented in Fig. 8.
point cloud attribute lossy compression. Compared to HAC,
Each figure presents the decoded point cloud with the total
our scheme obtains an average of 21.78%, 32.54%, and
attribute bitrate and the PSNR for the luma component. The
26% BD-BR gains on coding Y, U, and V components.
proposed scheme can obtain the highest PSNR values at the
Compared to BAAC, we can averagely get 3.92%, 15.98%,
lowest bitrate expenses compared to comparative platforms.
and 10.49% BD-BR gains on Y, U, and V components. HAC
Besides, we visualize the mean square error (MSE) of the
and BAAC are two typical graph transform based attribute
luma component for all points with error color-coding maps.
lossy codecs, while our scheme can beyond the limitations of
These maps effectively display the attribute distortion between
the graph and achieve better coding performance. Compared
the original and decoded point clouds. Results show that the
to the latest standard platform G-PCC (v23), an average of
proposed scheme can achieve better subjective and objective
14.14%, 17.52%, and 17.87% BD-BR gains can be achieved
quality compared to other platforms in point cloud attribute
on coding Y, U, and V components. The coding performance
lossy compression.
improvement over G-PCC is achieved in the proposed scheme
by leveraging all neighbors from different directions in a
diffusion-based iterative interpolation process. R-D curves C. Ablation Studies
with total attribute bitrates and the combined PSNR values of We perform a set of ablation studies to objectively ver-
the proposed scheme and comparative platforms on point cloud ify the effectiveness of our key modules in the proposed
attribute lossy compression are presented in Fig. 6. Consistent attribute compression framework. Validation results on point
R-D performance gains can be obtained by the proposed cloud attribute lossy compression are presented in Table II.
scheme on the vast majority of datasets at different bitrate We independently discuss each module as follows.
points. Results demonstrate that the proposed attribute predic- 1) Validity of PDE-Based Attribute Interpolation: The pro-
tive coding scheme can handle different types of point clouds posed PDE-based attribute interpolation module illustrated in
and achieve significant coding performance improvement over Section IV-C can generate a reconstructed point cloud with
the state-of-the-art methods at different bitrate points. smoothly interpolated attributes, where attributes of non-seed
2) Reconstruction Quality Evaluation: To further verify the points are inpainted from decoded seeds in a diffusion process.
advancement of the proposed scheme on point cloud attribute The module can effectively improve the attribute recon-
reconstruction quality after lossy compression, we conduct struction quality by fully utilizing the attribute correlation
subjective and objective quality evaluations on decoded point in neighboring regions. Table II shows that our attribute

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
SHAO et al.: 3D POINT CLOUD ATTRIBUTE COMPRESSION 9643

Fig. 6. R-D performance comparison of the proposed method, HAC, BAAC, and G-PCC on point cloud attribute lossy compression.

Fig. 7. Subjective quality comparison of decoded point clouds between our scheme and the state-of-the-art methods on the Soldier and T haidancer . Each
sub-figure presents the reconstructed attribute details in the red box, as well as the associated attribute bitrate (bpp) and PSNR (dB) for the luma component.
Local details within the yellow circle can display the quality differences of decoded point clouds.

interpolation module can averagely bring 5.77%, 9.55%, and we disable the anisotropic diffusion by using an identity array
10.25% BD-BR gains on the coding of Y, U, and V compo- as a diffusion tensor. Table II demonstrates that the EED
nents. Results validate the advantage of the proposed attribute module yields BD-BR gains of 5.55%, 2.44%, and 2.26%
interpolation module in the proposed framework. It outper- for the attribute lossy coding of Y, U, and V components,
forms the interpolation-based prediction in G-PCC, which respectively. Results demonstrate the benefits of the proposed
directly uses decoded neighboring attributes as prediction EED module, which can effectively improve the accuracy of
references. attribute prediction.
2) Validity of EED-Based Attribute Prediction: The pro- 3) Validity of Attribute Iterative Inpainting Strategy With
posed EED-based attribute prediction module, as illustrated Adaptive Early Termination: As shown in Fig. 4, we design an
in Section IV-D, can generate texture-aware attribute predic- iterative inpainting strategy with adaptive early termination to
tors. In this approach, edge-enhancing diffusion tensors are efficiently perform progressive attribute interpolation. To con-
extracted from local regions to effectively capture the under- firm the effectiveness of this scheme, we present the attribute
lying attribute structure. To validate the effectuality of EED, interpolation quality at different iterations in Fig. 9. As the

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
9644 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 34, NO. 10, OCTOBER 2024

Fig. 8. Objective quality comparison of decoded point clouds between our scheme and the state-of-the-art methods on the Soldier and T haidancer . The
mean square errors of the luma component between the original and decoded point clouds are quantitatively visualized with error color-coding maps.

TABLE II
A BLATION S TUDIES ON K EY M ODULES OF T HE P ROPOSED F RAMEWORK ON P OINT C LOUD ATTRIBUTE L OSSY C OMPRESSION

Fig. 9. Demonstration on the effectiveness of the proposed attribute iterative inpainting scheme on T haidancer and Longdr ess. The first row presents the
reconstructed point cloud with inpainted attributes at different iterations, and the second row presents the point cloud composed of points to be updated at
different iterations. As the iteration increases, the quality of the reconstructed point cloud increases, while the number of points to be updated decreases.

number of iterations increases, the quality of the reconstructed be updated gradually decreases. To avoid dispensable iterations
point cloud gradually improves, while the number of points to at smooth interpolated regions, we use the number of points to

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
SHAO et al.: 3D POINT CLOUD ATTRIBUTE COMPRESSION 9645

be updated after the iteration as the indicator for the proposed [9] B. Zhao, W. Lin, and C. Lv, “Fine-grained patch segmentation and
adaptive early termination. Table II shows that the proposed rasterization for 3-D point cloud attribute compression,” IEEE Trans.
Circuits Syst. Video Technol., vol. 31, no. 12, pp. 4590–4602, Dec. 2021.
adaptive early termination can obtain 65.00% encoder time and [10] H. Liu, H. Yuan, Q. Liu, J. Hou, H. Zeng, and S. Kwong, “A hybrid
65.66% decoder time reduction on attribute lossy compression. compression framework for color attributes of static 3D point clouds,”
Meanwhile, 0.17% BD-BR gains are averagely obtained on the IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 3, pp. 1564–1577,
Mar. 2022.
lossy coding of the Y component, while a marginal BD-BR [11] S. Gu, J. Hou, H. Zeng, and H. Yuan, “3D point cloud attribute
loss like 0.03% on the U component and 0.08% on the V compression via graph prediction,” IEEE Signal Process. Lett., vol. 27,
component is introduced by the proposed early termination. pp. 176–180, 2020.
This is an acceptable loss in the performance tradeoff between [12] S. Gu, J. Hou, H. Zeng, H. Yuan, and K.-K. Ma, “3D point cloud
attribute compression using geometry-guided sparse representation,”
the compression ratio and computational complexity. The IEEE Trans. Image Process., vol. 29, pp. 796–808, Aug. 2019, doi:
results manifest the effectiveness of the proposed iterative 10.1109/TIP.2019.2936738.
inpainting strategy with adaptive early termination. [13] Y. Shao, Z. Zhang, Z. Li, K. Fan, and G. Li, “Attribute compression of
3D point clouds using Laplacian sparsity optimized graph transform,”
IEEE Vis. Commun. Image Process. (VCIP), pp. 1–4, Dec. 2017.
VI. C ONCLUSION [14] G-PCC Test Model V23, Standard ISO WG7N00645, MPEG 3D Graph.
Coding, 2023.
In this paper, we propose a diffusion-based attribute [15] L. Wei, S. Wan, F. Yang, and Z. Wang, “Content-adaptive level of
progressive prediction framework for point cloud attribute detail for lossless point cloud compression,” APSIPA Trans. Signal Inf.
compression. Instead of the linear interpolation-based predic- Process., vol. 11, no. 1, pp. 1–28, 2022.
[16] J. Chen, L. Yu, and W. Wang, “Hilbert space filling curve based
tion paradigm, we solve the discrete attribute prediction as the scan-order for point cloud attribute compression,” IEEE Trans. Image
continuous diffusion-based interpolation, which can achieve Process., vol. 31, pp. 4609–4621, 2022.
accurate intra prediction and improve coding performance. [17] F. Song, G. Li, X. Yang, W. Gao, and S. Liu, “Block-adaptive point
PDE operators are designed to conduct optimized attribute cloud attribute coding with region-aware optimized transform,” IEEE
Trans. Circuits Syst. Video Technol., vol. 33, no. 8, pp. 1–12, Jul. 2023.
interpolation from seeds to unknowns, fully utilizing attribute [18] I. Galić, J. Weickert, M. Welk, A. Bruhn, A. Belyaev, and H.-P. Seidel,
spatial correlation in neighboring regions. An iterative update “Image compression with anisotropic diffusion,” J. Math. Imag. Vis.,
strategy with early termination is proposed to guarantee the vol. 31, nos. 2–3, pp. 255–269, Jul. 2008.
[19] P. Peter and J. Weickert, “Colour image compression with anisotropic
effectiveness and efficiency of the diffusion-based attribute diffusion,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Oct. 2014,
interpolation. Moreover, an EED-based attribute prediction is pp. 4822–4826.
devised to perform texture-aware intra prediction with inter- [20] P. Peter, L. Kaufhold, and J. Weickert, “Turning diffusion-based image
polated attributes, which can obtain smooth prediction in local colorization into efficient color compression,” IEEE Trans. Image Pro-
cess., vol. 26, no. 2, pp. 860–869, Feb. 2017.
regions while edge details can be preserved. Experimental [21] R. Schnabel and R. Klein, “Octree-based point-cloud compression,” PBG
results demonstrate that our proposed framework outperforms SIGGRAPH, vol. 3, 2006, pp. 111–121.
several competitive platforms in terms of compression perfor- [22] Y. Huang, J. Peng, C.-C. J. Kuo, and M. Gopi, “A generic scheme for
progressive point cloud coding,” IEEE Trans. Vis. Comput. Graphics,
mance and attribute reconstruction quality. Besides, ablation vol. 14, no. 2, pp. 440–453, Mar. 2008.
studies can prove the effectiveness of key modules in our [23] C. Zhang, D. Florencio, and C. Loop, “Point cloud attribute compression
scheme. with graph transform,” in Proc. IEEE Int. Conf. Image Process. (ICIP),
Oct. 2014, pp. 2066–2070.
[24] R. A. Cohen, D. Tian, and A. Vetro, “Attribute compression for sparse
R EFERENCES point clouds using graph transforms,” in Proc. IEEE Int. Conf. Image
[1] Q. Wang and M.-K. Kim, “Applications of 3D point cloud data in the Process. (ICIP), Sep. 2016, pp. 1374–1378.
construction industry: A fifteen-year review from 2004 to 2018,” Adv. [25] Y. Shao, Q. Zhang, G. Li, Z. Li, and L. Li, “Hybrid point cloud attribute
Eng. Informat., vol. 39, pp. 306–319, Jan. 2019. compression using slice-based layered structure and block-based intra
[2] P. Gao, S. Luo, and M. Paul, “Rate-distortion modeling for bit rate prediction,” in Proc. 26th ACM Int. Conf. Multimedia, Oct. 2018,
constrained point cloud compression,” IEEE Trans. Circuits Syst. Video pp. 1199–1207.
Technol., vol. 33, no. 5, pp. 2424–2438, May 2023. [26] Y. Xu et al., “Predictive generalized graph Fourier transform for attribute
[3] Z. Wei, B. Niu, H. Xiao, and Y. He, “Isolated points prediction via compression of dynamic point clouds,” IEEE Trans. Circuits Syst. Video
deep neural network on point cloud lossless geometry compression,” Technol., vol. 31, no. 5, pp. 1968–1982, May 2021.
IEEE Trans. Circuits Syst. Video Technol., vol. 33, no. 1, pp. 407–420, [27] R. L. de Queiroz and P. A. Chou, “Compression of 3D point clouds using
Jan. 2023. a region-adaptive hierarchical transform,” IEEE Trans. Image Process.,
[4] D. T. Nguyen and A. Kaup, “Lossless point cloud geometry and attribute vol. 25, no. 8, pp. 3947–3956, Aug. 2016.
compression using a learned conditional probability model,” IEEE Trans. [28] Y. Shen, W. Dai, C. Li, J. Zou, and H. Xiong, “Multi-scale structured
Circuits Syst. Video Technol., vol. 33, no. 8, pp. 4337–4348, Jan. 2023, dictionary learning for 3-D point cloud attribute compression,” IEEE
doi: 10.1109/TCSVT.2023.3239321. Trans. Circuits Syst. Video Technol., vol. 31, no. 7, pp. 2792–2807,
[5] X. Zhang and W. Gao, “Adaptive geometry partition for point cloud Jul. 2021.
compression,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 12, [29] F.-A. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah, “Diffusion
pp. 4561–4574, Dec. 2021. models in vision: A survey,” IEEE Trans. Pattern Anal. Mach. Intell.,
[6] F. Song, Y. Shao, W. Gao, H. Wang, and T. Li, “Layer-wise geometry vol. 45, no. 9, pp. 10850–10869, Sep. 2023.
aggregation framework for lossless LiDAR point cloud compres- [30] J. Ho et al., “Cascaded diffusion models for high fidelity image gener-
sion,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 12, ation,” J. Mach. Learn. Res., vol. 23, pp. 2249–2281, Jan. 2022.
pp. 4603–4616, Dec. 2021. [31] S.-I. Cheng, Y.-J. Chen, W.-C. Chiu, H.-Y. Tseng, and H.-Y. Lee,
[7] W. Zhu, Y. Xu, D. Ding, Z. Ma, and M. Nilsson, “Lossy point cloud “Adaptively-realistic image generation from stroke and sketch with
geometry compression via region-wise processing,” IEEE Trans. Circuits diffusion model,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis.,
Syst. Video Technol., vol. 31, no. 12, pp. 4575–4589, Dec. 2021. Jul. 2023, pp. 4054–4062.
[8] Y. Shao, G. Li, Q. Zhang, W. Gao, and S. Liu, “Nonrigid registration- [32] A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and
based progressive motion compensation for point cloud geometry L. van Gool, “RePaint: Inpainting using denoising diffusion probabilis-
compression,” IEEE Trans. Geosci. Remote Sens., vol. 61, 2023, doi: tic models,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.
10.1109/TGRS.2023.3321289. (CVPR), Jun. 2022, pp. 11461–11471.

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.
9646 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 34, NO. 10, OCTOBER 2024

[33] O. Elharrouss, N. Almaadeed, S. Al-Maadeed, and Y. Akbari, “Image Xiaodong Yang received the B.S. degree from
inpainting: A review,” Neural Process. Lett., vol. 51, pp. 2007–2028, Dalian University of Technology in 2021. He is cur-
2020. rently pursuing the master’s degree with Shenzhen
[34] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpaint- Graduate School, Peking University. He was
ing,” in Proc. 27th Annu. Conf. Comput. Graph. Interact. Techn., 2000, enrolled with the School of Electronic and Computer
pp. 417–424. Engineering, Shenzhen Graduate School, Peking
[35] M. Mainberger, A. Bruhn, J. Weickert, and S. Forchhammer, “Edge- University, in 2021. His research interests include
based compression of cartoon-like images with homogeneous diffusion,” image/video/3D point cloud compression.
Pattern Recognit., vol. 44, no. 9, pp. 1859–1873, Sep. 2011.
[36] C. Schmaltz, P. Peter, M. Mainberger, F. Ebel, J. Weickert, and
A. Bruhn, “Understanding, optimising, and extending data compression
with anisotropic diffusion,” Int. J. Comput. Vis., vol. 108, no. 3,
pp. 222–240, Jul. 2014.
[37] J. Liang, R. Lai, T. W. Wong, and H. Zhao, “Geometric understanding Wei Gao (Senior Member, IEEE) received the
of point clouds using laplace-beltrami operator,” in Proc. IEEE Conf. Ph.D. degree in computer science from the City
Comput. Vis. Pattern Recognit., Jun. 2012, pp. 214–221. University of Hong Kong in 2017. Since 2019,
he has been an Assistant Professor with the School
[38] R. Lai, J. Liang, and H.-K. Zhao, “A local MESH method for solv-
of Electronic and Computer Engineering, Shenzhen
ing PDEs on point clouds,” Inverse Problems Imag., vol. 7, no. 3,
Graduate School, Peking University, China. His
pp. 737–755, 2013.
research interests include image and video cod-
[39] A. Elmoataz, O. Lezoray, and S. Bougleux, “Nonlocal discrete regu-
ing, point cloud compression and processing, visual
larization on weighted graphs: A framework for image and manifold
perception modeling, deep learning, and artificial
processing,” IEEE Trans. Image Process., vol. 17, no. 7, pp. 1047–1060,
intelligence.
Jul. 2008.
[40] F. Lozes, A. Elmoataz, and O. Lézoray, “Partial difference operators on
weighted graphs for image processing on surfaces and point clouds,”
IEEE Trans. Image Process., vol. 23, no. 9, pp. 3896–3909, Sep. 2014.
[41] R. Hooda and W. D. Pan, “Early termination of dyadic region-adaptive Shan Liu (Fellow, IEEE) received the B.Eng. degree
hierarchical transform for efficient attribute compression of 3D point in electronic engineering from Tsinghua University
clouds,” IEEE Signal Process. Lett., vol. 29, pp. 214–218, 2022. and the M.S. and Ph.D. degrees in electrical engi-
[42] J. Weickert, Anisotropic Diffusion in Image Processing, vol. 1. Stuttgart, neering from the University of Southern California.
Germany: B. G. Teubner (Stuttgart), 1998. She was the Director of the Media Technology Divi-
[43] P. Charbonnier, L. Blanc-Feraud, G. Aubert, and M. Barlaud, “Determin- sion, MediaTek, USA. She was also with MERL,
istic edge-preserving regularization in computed imaging,” IEEE Trans. Sony, and IBM. She is currently a Tencent Distin-
Image Process., vol. 6, no. 2, pp. 298–311, Feb. 1997. guished Scientist and the General Manager of the
[44] Common Test Conditions for G-PCC, Standard ISON00650, MPEG 3D Media Laboratory, Tencent. She has been actively
Graph. Coding, 2023. contributing to international standards since the last
[45] C. Loop, Q. Cai, S. O. Escolano, and P. A. Chou, 8I Voxelized Full decade and served as co-editor of HEVC SCC and
Bodies? A Voxelized Point Cloud Dataset, Standard ISOm38673, 2016. the emerging VVC. She has numerous technical contributions adopted into
[46] G. Bjøntegaard, Calculation of Average PSNR Differences Between RD- various standards, such as HEVC, VVC, OMAF, DASH, and PCC. At the
curves, Standard VCEG-M33, Video Coding Experts Group, Austin, same time, technologies and products developed by her and under her
Texas, USA, 2001. leadership have served several hundred million users. She holds more than
150 granted U.S. and global patents and has published more than 80 journal
and conference papers. Her research interests include audio-visual, high-
volume, immersive, and emerging media compression, intelligence, transport,
and systems. She was appointed as the Vice Chair of the IEEE Data
Compression Standards Committee in 2019. She was on the Committee of
Industrial Relationship of the IEEE Signal Processing Society from 2014 to
2015 and on the Editorial Board of IEEE T RANSACTIONS ON C IRCUITS AND
S YSTEMS FOR V IDEO T ECHNOLOGY from 2018 to 2021. She was the VP
of the Industrial Relations and Development of the Asia-Pacific Signal and
Information Processing Association from 2016 to 2017 and was named the
APSIPA Industrial Distinguished Leader in 2018.

Yiting Shao received the B.S. degree from the

Minzu University of China in 2016. She is currently
pursuing the Ph.D. degree with the School of Elec- Ge Li (Member, IEEE) is currently a Professor with
tronics Engineering and Computer Science, Peking the School of Electronic and Computer Engineer-
University. She has been actively contributing to ing, Shenzhen Graduate School, Peking University,
international standards developed by the Moving China. His research interests include image/video
Picture Experts Group (MPEG) and the Audio Video process and analysis, machine learning, digital com-
Coding Standard Working Group of China (AVS) munications, and signal processing.
since 2017. She has several technical contributions
adopted into various standards, such as MPEG
PCC and AVS PCC. Her research interests include
image/video/3D point cloud compression.

Authorized licensed use limited to: University of Southern California. Downloaded on December 22,2024 at 10:32:18 UTC from IEEE Xplore. Restrictions apply.