7618 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 32, NO.

11, NOVEMBER 2022

No-Reference Quality Assessment for 3D Colored

Point Cloud and Mesh Models
Zicheng Zhang , Student Member, IEEE, Wei Sun , Xiongkuo Min , Member, IEEE,
Tao Wang , Wei Lu , and Guangtao Zhai , Senior Member, IEEE

Abstract— To improve the viewer’s Quality of Experi- scenarios such as virtual reality (VR), medical 3D reconstruc-
ence (QoE) and optimize computer graphics applications, tion, and video post-production [1], etc. Among the digital
3D model quality assessment (3D-QA) has become an important representation forms of 3D models, point cloud and mesh
task in the multimedia area. Point cloud and mesh are the
are the most widely used formats in practical. A point cloud
two most widely used digital representation formats of 3D
models, the visual quality of which is quite sensitive to lossy based object is a set of points in space, in which each point is
operations like simplification and compression. Therefore, many described with geometry coordinates and sometimes with other
related studies such as point cloud quality assessment (PCQA) attributes such as color and surface normals. Mesh is more
and mesh quality assessment (MQA) have been carried out to complicated because it is a collection of vertices, edges, and
measure the visual quality of distorted 3D models. However, faces which together define the shape of a 3D model. Except
most previous studies utilize full-reference (FR) metrics, which for geometry information, the 3D mesh may also contain
indicates they can not predict the quality level in the absence
other appearance attributes, such as color and material. Both
of the reference 3D model. Furthermore, few 3D-QA metrics
consider color information, which significantly restricts their point cloud and mesh are able to vividly display exquisite
effectiveness and scope of application. In this paper, we pro- models and complex scenes. However, since point cloud and
pose a no-reference (NR) quality assessment metric for colored mesh record the omnidirectional details of objects and scenes,
3D models represented by both point cloud and mesh. First, lossless 3D models usually need large storage space and very
we project the 3D models from 3D space into quality-related high transmission bandwidth in practical applications. Hence,
geometry and color feature domains. Then, the 3D natural scene a variety of 3D processing algorithms such as simplification
statistics (3D-NSS) and entropy are utilized to extract quality-
aware features. Finally, a support vector regression (SVR) model
and compression, etc. have been proposed to satisfy the
is employed to regress the quality-aware features into visual specific needs, which inevitably cause damage to the visual
quality scores. Our method is validated on the colored point quality of 3D models [2], [3]. Additionally, in some 3D
cloud quality assessment database (SJTU-PCQA), the Waterloo scanning model APIs like Apple 3D object capture [4] and
point cloud assessment database (WPC), and the colored mesh Intel Lidar Camera Realsense [5], some slight disturbance
quality assessment database (CMDM). The experimental results such as blur, noise, etc. may be introduced to the constructed
show that the proposed method outperforms most compared NR 3D models.
3D-QA metrics with competitive computational resources and
greatly reduces the performance gap with the state-of-the-art Therefore, to improve users’ Quality of Experience (QoE)
FR 3D-QA metrics. The code of the proposed model is publicly in 3D fields and optimize 3D compression and reconstruction
available now at https://github.com/zzc-1998/NR-3DQA. systems, it is of great significance to develop quality metrics
for point cloud quality assessment (PCQA) and mesh quality
Index Terms— 3D model quality assessment, colored point
cloud, colored mesh, no-reference quality assessment, natural assessment (MQA). However, different from the format of 2D
scene statistics. media like images and videos where pixels are distributed
in the fixed grid, the points in 3D models are distributed
I. I NTRODUCTION irregularly in the space, which brings a huge challenge for
3D-QA tasks. For example, the neighborhood of pixels in
N OWADAYS, with the rapid development of computer
graphics, the digital representation of 3D models has
been widely studied and used in a wide range of application
2D media can be easily obtained, thus the local features
are available by analyzing the relationship between the pixel
and its neighborhood. However, the neighborhood for points
Manuscript received 21 February 2022; revised 2 May 2022; in 3D models is very ambiguous, which makes it difficult
accepted 22 June 2022. Date of publication 28 June 2022; date of current to conduct local feature analysis. Although some 3D-QA
version 28 October 2022. This work was supported in part by the National
Science Foundation of China under Grant 61831015 and in part by the
databases [6]–[10] have been proposed to push forward the
National Key Research and Development Program of China under Grant development of 3D-QA algorithms, the difficulty of collecting
2021YFE0206700. This article was recommended by Associate Editor 3D models and conducting subjective experiments greatly limit
L. Agostini. (Corresponding author: Guangtao Zhai.) the size of the 3D-QA databases, which may also restrict
The authors are with the Institute of Image Communication and
Network Engineering, Shanghai Jiao Tong University, Shanghai 200240, relevant research, especially for NR 3D-QA metrics.
China (e-mail: [email protected]; [email protected]; minxiongkuo@
sjtu.edu.cn; [email protected]; [email protected]; A. Previous 3D-QA Works
[email protected]).
Color versions of one or more figures in this article are available at Quality assessment can be divided into subjective quality
https://doi.org/10.1109/TCSVT.2022.3186894. assessment and objective quality assessment according to
Digital Object Identifier 10.1109/TCSVT.2022.3186894 whether human observers are involved. It is known that,
1051-8215 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 ZHANG et al.: NR QUALITY ASSESSMENT FOR 3D COLORED POINT CLOUD AND MESH MODELS 7619

in subjective quality assessment, large numbers of people 2) The Development of MQA: The MQA metrics can be
are required to assess the visual quality of 3D models and categorized into two types: model-based metrics [14], [15],
give their subjective scores. Although the mean opinion [19], [30], [31] which operate directly on the 3D models,
scores (MOSs) collected from human observers are straightfor- and IQA-based metrics [21], [22], [32]–[34] which operate on
ward and precise, they cannot be used in practical applications the rendering snapshots of 3D models. Supported by the vast
due to the huge time consumption and high expense [11]. amount of previous research on FR-IQA methods [35]–[37],
Therefore, objective quality metrics are urgently needed to many FR-MQA models have been proposed using similar
predict the visual quality of 3D models automatically. The manners, which usually compute local features at the vertex
3D objective quality assessment can then be divided into level and then pool the features into a quality value. For exam-
full-reference (FR), reduced-reference (RR), and no-reference ple, MSDM2 [14] uses the differences of structure (captured
(NR) 3D-QA methods. FR 3D-QA methods work by com- via curvature statistics) computed on local neighborhoods to
paring the difference between the reference 3D models and predict the quality level. DAME [19] measures the differences
distorted 3D models, RR 3D-QA methods employ part of the in dihedral angles between the reference and the distorted
reference models’ information for comparison, and NR 3D-QA meshes to evaluate the quality loss. FMPD [15] estimates the
methods only analyze the distorted 3D models to give the local roughness difference derived from Gaussian curvature
perceptual quality scores. Considering the complexity of 3D to assess the quality of the distorted mesh. However, these
models, in the literature, a large part of 3D-QA metrics are metrics only take geometry information into consideration.
full-reference and they only take geometry features into con- Furthermore, to analyze the influence of color information,
sideration [12]–[19]. When it comes to 3D models with color some color-involved FR-MQA metrics have been carried out.
information, limited works have been proposed [20]–[25]. Tian and AlRegib [21] introduce a global distance over texture
Clearly, the NR 3D-QA for colored models has fallen behind. image using Mean Squared Error (MSE) to quantify the effect
In this section, we briefly review the development of 3D-QA of color information. Guo et al. [22] exploit SSIM to calculate
and introduce the mainstream methods designed for the texture image distance as the color information features.
3D-QA tasks. Nehmé et al. [7] introduce a metric to incorporate perceptually
1) The Development of PCQA: The earliest FR-PCQA relevant curvature-based and color-based features to evaluate
metrics usually focus on the geometry aspect at the point the visual quality of colored meshes.
level, such as p2point [12], p2plane [13], and p2mesh [16]. Recently, thanks to the effectiveness of machine learning
The p2point estimates the levels of distortion by computing the technologies, some learning-based NR-MQA metrics have
distance vector between the corresponding points. The p2plane been proposed. Abouelaziz et al. [30] extract features using
further projects the distance vector on the normal orientation dihedral angle models and train a support vector machine
for evaluating the quality loss. The p2mesh first reconstructs for feature regression. Later, Abouelaziz et al. [31] scale the
the point cloud to mesh and then measures the distance from curvature and dihedral angle into 2D patches and utilize the
points to the reconstructed surface to predict the quality level, convolution neural network (CNN) for training. They further
however, it greatly depends on the reconstruction algorithms introduce a CNN framework with saliency views rendered
and lacks stability. Since the point-level difference is difficult from 3D meshes [32]. However, the NR methods mentioned
to reflect the complex structural distortions, some studies above only extract geometry features and may fail to accu-
further consider other structural characteristics for PCQA. rately predict the scores of colored meshes.
For example, Alexiou and Ebrahimi [17] adopt the angular
difference of point normals to estimate the degradations.
B. Our Approach
Javaheri et al. [18] utilize the generalized Hausdorff distance
to reflect the distortions caused by compression operations. Generally speaking, a large part of the metrics mentioned
In some situations, color information can not be ignored, above are full-reference metrics, which can make full use of
which challenges the PCQA methods considering only geom- the relationship between the reference and distorted 3D models
etry information. To incorporate the color information into and give relatively accurate results. However, the disadvantage
PCQA models, Meynet et al. [23] propose a metric to use of FR 3D-QA metrics is also obvious. They are not able to
the weighted linear combination of curvature and color inf work in the absence of reference 3D models. Unfortunately,
ormation to evaluate the visual quality of distorted point in many application scenarios like 3D reconstruction, pristine
clouds. Inspired by SSIM [26] and Alexiou and Ebrahimi [20] reference is not always available. Therefore, inspired by the
compute the similarity of four types of features, including huge success of natural scene statistics (NSS) in IQA tasks [9],
geometry, normal vectors, curvature, and color information. [38], [39], some researchers [30], [40], [41] use similar ideas
What’s more, some studies [6], [9] try to predict the quality of extracting handcrafted features and estimating the statistical
level by evaluating the projected 2D images from 3D models. parameters with certain NSS distributions as quality-aware
The advantage is that image quality assessment (IQA) metrics information. In this paper, we further push forward the applica-
have been well developed while the disadvantage is that there tion of NSS for both colored point cloud and mesh models, and
is inevitable information loss during the projection and the we summarize such a concept as 3D-NSS. Specifically, NSS is
projected images are easily affected by the projected angles a discipline within the field of perception, which is dependent
and viewpoints. on the premise that the perceptual system is designed to
Few NR-PCQA metrics have been developed so far, which interpret natural scenes [39]. Through observations of feature
include a learning-based approach [27] using two modified distributions, we find that the reference features obey certain
PointNet [28] as feature extraction backbone and also a NSS distributions and different types of distortions change the
learning-based method [29] using multi-view projection. appearance of such feature distributions. Therefore, we believe
Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 7620 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 32, NO. 11, NOVEMBER 2022

Fig. 1. The framework of the proposed method. Geometry-based and color-based features are first extracted from the distorted 3D models. Then various
statistical parameters are estimated from the extracted features to form the feature vector. Finally, the quality scores are given through the SVR regression
module.

that 3D-NSS is effective to quantify the visual quality of Section IV presents the experiment setup and the experimental
3D models in the presence of distortion. More details of results. Section V summarizes of this paper.
3D-NSS are discussed in Section III.
Specifically, we first project the 3D models into
quality-related geometry and color feature domains. Then, II. F EATURE P ROJECTION
3D-NSS and entropy are utilized to extract these color and The framework of the proposed method is illustrated in
geometry characteristics. Finally, the obtained features are Fig. 1, which includes a feature extraction module, a parame-
integrated into a quality value through the support vector ters estimation module, and a regression module. Point cloud
regression (SVR) model. In order to test the effectiveness of and mesh have similar quality-aware geometric properties as
different types of features and different kinds of distribution well as the attached color attributes. Therefore, we uniformly
models, we test the performance of various combinations of design the point cloud and mesh processing framework, and
the statistical parameters to find the optimal combination. Fur- determine the specific projection domains based on their char-
ther, we conduct the data-sensitivity experiment, the ablation acteristics. Before introducing the proposed model, we first
study, and the computational efficiency experiment to demon- define a distorted 3D object O as:
strate the effectiveness of our method. In-depth discussions are
given as well. O ∈ {P, M},
P = {Poi nts},
C. Contributions of This Paper M = {V er ti ces, Edges, Faces}, (1)
We summarize our contributions as follows: where P and M mean that the 3D object is represented by
• We push forward the development of NSS in the 3D-QA point cloud and mesh respectively, and the color information
fields based on previous research [30], [40], [41] and we is attached to Poi nts in point cloud and V er ti ces in mesh
systematically summarize the concept as 3D-NSS. respectively.
• We propose a no-reference quality assessment metric for
both colored point cloud and mesh. We extract features
not only from the geometry aspect, but also from the color A. Geometry Feature Projection
aspect. Furthermore, it is the first method that can deal Geometry features usually have a strong correlation with
with both NR-PCQA and NR-MQA with color informa- human perception, which has been firmly proved in 3D-QA
tion. The proposed NR 3D-QA model follows a common studies [12]–[19]. Although the geometry features are com-
NR framework, which means it is easy to modify our puted in different ways for point cloud and mesh, they share
metric for performance improvement or meeting other similar characteristics for the visual quality of 3D models.
specific needs. In this section, we project the given 3D model into several
• We deeply investigate the effectiveness of different types quality-aware geometry feature domains:
of features and different kinds of NSS models, which can
provide useful guidelines for future research. Fgeo = Projectiongeo(O),
• Compared with the state-of-the-art methods, our method O ∈ {P, M}, (2)
is more efficient in the computation process, which means
the proposed method is potentially more capable of where Fgeo indicates the set of geometry feature domains of
handling practical situations. The code of the proposed the 3D model, Projectiongeo(·) denotes the geometry projec-
model is also released for promoting the development tion function.
of NR 3D-QA. 1) Point Cloud Geometry Feature Domains: Considering
The paper is organized as follows. Section II describes that the point cloud lacks surfaces, we first need to get the
the feature projection processes. Section III describes the neighborhood set for each point so we can further extract
process of quantifying the distortions to statistical parameters. the geometry features. Given the point cloud P = { pi }i=1
N
,

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 ZHANG et al.: NR QUALITY ASSESSMENT FOR 3D COLORED POINT CLOUD AND MESH MODELS 7621

the neighborhood PNbi of each vertex pi can be obtained

utilizing the k-nearest neighbors (k-NN) algorithm:
PNb = KNN(P),
k=10

 2
Di st ( p, q) = ( p x −qx )2 + p y −q y +( pz −qz )2 , (3)
where N indicates the number of points in the point cloud,
PNb is the set of all neighborhoods, KNN(·) denotes the k-NN
algorithm function, the k level is set as 10, and Euclidean Fig. 2. Example of the variables corresponding to the curvature calculation,
where the red line indicates |e ∩ B| and C(e) indicates the curvature
distance Di st ( p, q) is exploited as the distance function. contribution of e.
With the computed neighborhood PNbi , the corresponding
covariance matrix Ci for each point pi (represented by
λ2 − λ3
3D geometry coordinates) can be denoted as: Pl a( pi ) = , (9)
λ1
1 
K λ3
Ci = ( pn j − p̂)( pn j − p̂) , Sph( pi ) = , (10)
K λ1
j =1
where λ1 , λ2 , and λ3 refer to the corresponding eigenvalues
× { pn1 , . . . , pn K } ∈ PNbi , (4)
respectively. Through the operations mentioned above, the
where pn j and p̂ are vectors of dimension 3×1, Ci is a matrix point cloud is transformed into 5 geometry feature domains.
of dimension 3 × 3, K stands for the size of neighborhood 2) Mesh Geometry Feature Domains: Compared with point
PNbi , and p̂ represents the centroid of neighborhood PNbi . cloud, the 3D mesh is more complicated because it is a
Therefore, the eigenvector problem for the covariance collection of vertices, edges, and faces which together define
matrix Ci can be described as: the shape of a 3D model. Therefore, the feature extraction
process is a bit different from that in the point cloud.
Ci · vl = λl · vl , l ∈ {1, 2, 3}, (5) a) Curvature: Thanks to the edges and faces information
contained in mesh, the computation of curvature is more
where (λ1 , λ2 , λ3 ) indicate the eigenvalues, (v 1 , v 2 , v 3 precise than the point cloud. So far, various curvature defini-
represent the corresponding eigenvectors, and we assume that tions have been introduced for 3D meshes [45], [46], among
λ1 > λ2 > λ3 ). Thus, a total of three eigenvalues are obtained which the average curvature is proven to be able to stably
for each point pi in the point cloud P. describe the local structural information of 3D meshes. Hence,
In previous studies [42], [43], the eigenvalues mentioned a weighted average curvature method introduced in [46] is
above are used to calculate various geometry features for tasks adopted to measure the roughness of embedded surfaces in
like classification, simplification, and segmentation. These 3D meshes. Given the mesh M = {V, E, F}, the weighted
features have shown great ability to describe the complicated average curvature is calculated by averaging the curvature
geometry structure information and achieved outstanding per- contributions over a certain region B around each vertex v i :
formance. Hence, we selected several geometry features that
1 
may be helpful for point cloud quality assessment: Cur (v i ) = C(e)|e ∩ B|, (11)
• Curvature: Curvature is the amount by which a curve |B|
e∈E
deviates from being a straight line, which is frequently where Cur (v i ) represents the weighted average curvature for
used to describe roughness or smoothness. vertex v, region B is a sphere region centered at v and its
• Anisotropy: Anisotropy is used to exhibit variations in
radius is 1/100 of the bounding box of the 3D mesh, |B| stands
geometrical properties for different directions. for the surface area of region B. E is the set of edges that are
• Linearity: Linearity is the property for estimating the
linked to the vertex v, C(e) indicates the curvature contribution
similarity to a straight line. of e (the signed angle between the normals of the two oriented
• Planarity: Planarity is utilized to measure the similarity
triangles incident to edge e), |e ∩ B| denotes the weight which
to a planar surface. is defined as the length of e ∩ B (always between 0 and |e|).
• Sphericity: Sphericity is the measure of how closely the
An example of the variables corresponding to the curvature
shape of an object resembles that of a perfect sphere. calculation is shown in Fig. 2.
Moreover, all the geometry features described above rep- b) Dihedral angle: For a 3D mesh, the dihedral angle is
resent the local distribution patterns and are calculated at the angle between the normals of two adjacent faces, which
the point level, which indicates that each pi has its own has been utilized in [47] as an effective indicator for measuring
curvature, anisotropy, linearity, planarity, and sphericity. The the loss of the mesh simplification. Furthermore, [19] exploits
formulations [44] for geometry features are denoted as: oriented dihedral angles instead of the simple dot product of
λ3 normals to better distinguish the convex and concave angles.
C ur( pi ) = , (6) Inspired by similar ideas, we adopt the dihedral angles set to
λ1 + λ2 + λ3
λ1 − λ3 describe the visual quality of 3D meshes. Assuming that the
Ani( pi ) = , (7) vertices for two adjacent faces f 1 and f2 are {v1, v3, v4} and
λ1
{v2, v3, v4}, we have:
λ1 − λ2
Li n( pi ) = , (8) Di h f1 , f2 = arccos (n 1 · n 2 ) · sgn (n 1 · (v 2 − v 1 )) , (12)
λ1

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 7622 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 32, NO. 11, NOVEMBER 2022

where Di h f1 , f2 is the oriented dihedral angle between two illuminant. The f (·) function is described as:
adjacent faces f 1 and f 2 , n 1 and n 2 are the normals of f 1 √
and f 2 , sgn(·) denotes the signum function which is used to
3
t, if t > δ 3 ,
f (t) = t (16)
decide the orientation of the dihedral angle. 3δ 2
+ 29 , otherwise ,
4

c) Face area and angle: Area and angle are two simple
attributes of 3D mesh faces, which can be easily computed where δ is set as 29
6
. Finally, the LAB color channels are
by making use of the coordinates of the face’s vertices. computed as the color feature domains.
In the mesh smoothing algorithm proposed in [48], attributes
including face angle are used to predict the new location III. E STIMATING S TATISTICAL PARAMETERS
of the smoothed nodes. While in the 3D mesh encoding Through prior knowledge and observations of the corre-
method introduced in [49], face angle is utilized to instruct sponding feature distributions, we find that the characteristic
the compression of 3D meshes, which indicates that face angle statistical parameters of NSS models can be changed by the
is related to the quality of 3D meshes. Therefore, to further presence of distortion. Therefore, we choose entropy and
measure the visual quality degradation of 3D meshes, the face several NSS models including the generalized Gaussian distri-
area and angle are collected as feature sets. bution (GGD), the general asymmetric generalized Gaussian
Finally, the mesh is projected into 4 geometry feature distribution (AGGD), and the shape-rate Gamma distribution
domains: for parameters estimation to quantify the perceptual quality of
• Curvature: The weighted average curvature is used to 3D models.
describe the geometry characteristics like roughness or
smoothness.
• Dihedral Angle: Dihedral angle is employed as a useful
A. Basic Statistical Parameters
descriptor for measuring the caused degradations. For each set of features, we exploit the normalization
• Face Area & Angle: These two attributes are highly operation as the pre-processing:
correlated with the effectiveness of lossy operations like F − mean(F)
compression. F̂ = ,
std(F) + C
F ∈ {Fgeo , Fcol }, (17)
B. Color Feature Projection where F represents the feature domain, mean(·) is the average
Color is a significant aspect of visual quality assessment. function, std(·) denotes the standard deviation function, and
For a colored point cloud, the color is directly determined C is a small constant to avoid instability. Entropy is believed
by the color information of the point, while for a colored to be highly correlated with the quantization distortion. Fig. 3
mesh, the color of the surface is generally rendered by the presents the normalized probability distributions of LAB chan-
color information of the contained vertices. Besides, the color nels for color quantization distortion. It can be observed that
information in the 3D models is usually stored in the form with the increasing quantization levels, the corresponding dis-
of RGB channels. However, the RGB color space has been tributions become significantly sparse, meaning the number of
proven to have a poor correlation with human perception. distinct colors is reduced. Considering that some simplification
Therefore, we adopt the LAB color transformation as the color and compression algorithms usually introduce quantization
feature projection: operations to the 3D models, we decide to use entropy as one
of the quality-aware features. Then the entropy can be derived
Fcol = Projectioncol(O), as:
O ∈ {P, M}, (13)
E = Entropy( F̂),
where Fcol indicates the set of color feature domains of the ˆ , Fcol
F̂ ∈ { Fgeo ˆ }. (18)
3D model, Projectioncol(·) stands for the color projection
where Entropy(·) indicates the entropy function, Fgeoˆ and
function, and P and M represent the distorted point cloud
and mesh respectively. The detailed color transformation is ˆ are the normalized feature distributions for geometry and
Fcol
formulated as: color respectively.
⎡ ⎤ ⎡ ⎤⎡ ⎤
X 2.7688 1.7517 1.1301 R
⎣ Y ⎦ = ⎣ 1.0000 4.5906 0.0601 ⎦ ⎣ G ⎦ , (14) B. GGD Parameters
Z 0 0.0565 5.5942 B Fig. 4 shows one reference model and three distorted point
⎧   cloud samples with different distortion types (compression,
⎪
⎪ L = 116 f YYn − 16,
⎪
⎨      downsampling, geometry Gaussian noise). It can be observed
A = 500 f XXn − f YYn , (15) that the reference curvature and anisotropy distributions shown
⎪
⎪      in Fig. 4 (e, i) exhibit Gamma-like appearance while the
⎪
⎩ B = 200 f Y − f Z
Yn Zn , reference linearity and planarity distributions shown in Fig. 4
(m, q) tend to be Gaussian-like. Obviously, with different
where R, G, B represent the corresponding RGB color chan- distortion types, the shapes of the distributions are changed.
nels, X, Y, Z stand for the corresponding XYZ color channels, For example, with closer inspections, we can see that compres-
L, A, B denote the corresponding RGB color channels, and sion adds more weight to the tail of curvature and anisotropy
X n , Yn , Z n describe the specified white achromatic reference distributions shown in Fig. 4 (f, j) while downsampling makes

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 ZHANG et al.: NR QUALITY ASSESSMENT FOR 3D COLORED POINT CLOUD AND MESH MODELS 7623

Fig. 3. A comparison example for color distortion in the CMDM database [7]. (a) represents the snapshots of the reference 3D mesh while (b)-(e) stand for
the snapshots of the meshes with 4 increasing levels of color information quantization. (f), (k), (p) represent the normalized probability distributions of LAB
channels for (a) model, (g), (l), (q) represent the normalized probability distributions of LAB channels for (b) model, (h), (m), (r) represent the normalized
probability distributions of LAB channels for (c) model, (i), (n), (s) represent the normalized probability distributions of LAB channels for (d) model, and
(j), (o), (t) represented the normalized probability distributions of LAB channels for (e) model respectively.

curvature and anisotropy distributions shown in Fig. 4 (g, k) C. AGGD Parameters

more centered. What’s more, geometry Gaussian noise makes However, in some situations, the shape of the distribution
all the geometry distributions more smooth and adds more is not symmetrical. We can see clearly from Fig. 4 that the
weight to the tail of curvature and anisotropy distributions reference planarity distribution represented by Fig. 4 (q) has
shown in Fig. 4 (h, l) on both sides. a longer tail on the left while the planarity distribution
The GGD distribution is effective for capturing a broad represented by Fig. 4 (t) has a longer tail on the right.
spectrum of statistics and reflecting the changes in distribution What’s more, all types of distortions change weight of the
shapes, especially for the tail part [50]. To capture the corre- tails for curvature and anisotropy distributions represented by
sponding characteristics of 3D model distributions, we propose Fig. 4 (f, g, h, j, k, l). Therefore, we utilize the general
to use the generalized Gaussian distribution (GGD): asymmetric generalized Gaussian distribution (AGGD) model
to extract more detailed parameters from normalized feature
  α  distributions to describe the different spread extent in two
α |x|
GG D(x; α, β ) = 2
exp − , directions:
2β(1/α) β ⎧   v 
⎪ v −x
x ∈ {Fgeo , Fcol }, (19)   ⎪ ⎨(β +β )  1  exp − βl x < 0,
2
AGG D x; v, σl , σr =2 l r v
  v 
⎪ v
⎩(β +βr )  1  exp − βr x ≥ 0,
x
  ∞ α−1 −t ⎪
where β = σ (1/α)
(3/α) , (α) = 0 t e dt, α > 0 is the l v

gamma function, and the two estimated parameters (α, β 2 ) ˆ , Fcol

x ∈ { Fgeo ˆ }, (20)
indicate the shape and variance of the distribution. What’s      
1 1
 v  v
more, considering that the variance for normalized distribution where βl = σl   , βr
3
= σr  ,η
3
= (βr − βl ), the
is fixed as 1, we estimate the GGD parameters from distribu-  v  v
tions before normalization. parameter v decides the shape of the distribution, σl2 and σl2

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 7624 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 32, NO. 11, NOVEMBER 2022

Fig. 4. Examples of point cloud samples from SJTU-PCQA database [6]. (a) is the reference point cloud sample, while (b), (c), and (d) are 3 distorted
point cloud samples with different distortion types (compression, downsampling, geometry Gaussian noise). Some features’ (Cur, Ani, Lin, Pla) normalized
probability distributions are selected as examples. More specifically, (e), (i), (m), and (q) are the corresponding features’ normalized probability distributions
of (a) model, (f), (j), (n), and (r) are the corresponding features’ normalized probability distributions of (b) model, (g), (k), (o), and (s) are the corresponding
features’ normalized probability distributions of (c) model, (h), (l), (p), and (t) are the corresponding features’ normalized probability distributions of (d) model
respectively.

are scale parameters that refer to the spread extent on the parameter that can better fit the AGGD model stated in [38].
left and right sides of the distribution respectively, η is a Additionally, the AGGD can be recognized as the extension

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 ZHANG et al.: NR QUALITY ASSESSMENT FOR 3D COLORED POINT CLOUD AND MESH MODELS 7625

TABLE I
S UMMARY OF F EATURES E XTRACTED IN THE P ROPOSED M ETHOD

of GGD. When σl2 = σr2 , the AGGD turns into GGD. Finally, In our experiment, we propose to use the support vector
the four parameters (η, v, σl2 , σl2 ) are estimated to describe the machine regressor (SVR) as the regression model, which is a
characteristics of asymmetric distributions. common and effective choice to handle high dimensional data
in previous quality assessment research [30], [38]. We employ
the min-max normalization as the pre-processing to scale the
D. Gamma Parameters
features. Then the feature vector can be integrated into a
The reference curvature and anisotropy distributions shown quality score for evaluation. We employ the Python sklearn
in Fig. 4 (e, i) are similar to the shape of Gamma distribution. package [51] to implement the radial basis function (RBF)
Compression, downsampling, and Gaussian noise all change kernel SVR model with default settings.
the shape and scale of the corresponding distributions and such
distributions still exhibit Gamma-like appearance. Therefore,
we propose to use Gamma distribution parameters to quantify B. Experiment Setup
the distortions. The shape-rate Gamma model is formulated 1) Experiment Setup for PCQA: To test the performance of
as: the proposed method, we employ the subjective point cloud
β α x . α − 1e−β x assessment database (SJTU-PCQA) [6] and the Waterloo point
Gamma(x; α, β) = x > 0, cloud assessment database (WPC) proposed in [10].
(α)
The SJTU-PCQA database provides 420 point cloud sam-
ˆ , Fcol
x ∈ { Fgeo ˆ }, (21) ples distorted from 10 reference point clouds. Each reference
where α and β stands for the shape and rate parameters and point cloud is distorted with seven types of common distortions
α, β > 0. with six levels. Unfortunately, only 9 reference point clouds
and their corresponding distorted point cloud samples are now
available to the public, thus we can obtain 378 (9×6×7) point
E. Parameters Summary cloud samples for the experiment. Since the proposed approach
In summary, we collect the average, standard devia- requires a training procedure to calibrate the SVR model and
tion, and entropy values in the 3D model feature domains to avoid the influence of content overlap, we select 8 of the
as the fundamental features. Then we use 3 represen- 9 groups of point clouds as training set and leave the rest
tative distribution models to estimate 8 statistical para- 1 group as testing set. In order to ensure the validity of the
meters, including GGD (α, β 2 ), AGGD (η, v, σl2 , σl2 ), results, we exhaustively list all the C98 = 9 database separa-
Gamma (α, β), for all feature domains. Finally, consider- tions for the experiment and use the average performance as
ing that the colored point cloud has 8 feature domains the final experimental results. In addition, the MOSs collected
(Cur, Ani, Li n, Pla, Sph, L, A, B) and the colored mesh has in the SJTU-PCQA database are divided by 10 in the training
7 feature domains (Cur, Di h, Far, Fan, L, A, B), 88 (8×11) process to scale the MOSs to [0,1]. The predicted scores are
features are computed for a single colored point cloud and re-scaled for validation in the testing process.
77 (7 × 11) features are computed for a single colored The WPC database includes 20 high-quality source point
mesh respectively. The summary of features extracted in the clouds and creates 740 distorted point clouds using down-
proposed method is listed in Table I. sampling, Gaussian noise, and three types of compression.
Specifically, we maintain the same training set and testing set
IV. E XPERIMENT E VALUATION split as stated in [29]. Similarly, the MOSs provided in the
WPC database are divided by 100 in the training process and
A. Regression Model the predicted scores are re-scaled for validation in the testing
After the feature extraction process, a feature vector is process. The comparison PCQA metrics can be categorized
obtained to describe the characteristics of the 3D model. into two types:

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 7626 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 32, NO. 11, NOVEMBER 2022

TABLE II
P ERFORMANCE C OMPARISON W ITH C OMPETITORS ON THE SJTU-PCQA AND WPC D ATABASES

• Image-based metrics: These metrics evaluate the quality for geometry-only 3D meshes include NR-SVR [30],
of 3D models by assessing the quality of the corre- NR-GRNN [55], NR-CNN [31].
sponding 2D projections. Please refer to [6] for detailed
projection process and we use only RGB projections. C. Evaluation Criterion
The FR image-based metrics include PSNR, SSIM [26], Four mainstream consistency evaluation criteria are utilized
and PB-PCQA [6]. The NR image-based metrics include to compare the correlation between the predicted scores and
NIQE [52], BRISQUE [38], and PQA-net [29]. MOSs, which include Spearman Rank Correlation Coefficient
• Model-based metrics: Full-reference metrics operate dir- (SRCC), Kendall’s Rank Correlation Coefficient (KRCC),
ectly from the 3D model, which include GraphSIM [53], Pearson Linear Correlation Coefficient (PLCC), and Root
PointSSIM [20], PCQM [23], and ResCNN [27]. Mean Squared Error (RMSE). An excellent model should
Reduced-reference metric includes PCMRR [54]. Specifi- obtain values of SRCC, KRCC, and PLCC close to 1, and
cally, ResCNN is a model-based deep-learning approach. the value of RMSE near 0.
2) Experiment Setup for MQA: The MQA method proposed
in this paper is validated on the color mesh distortion mea- D. Performance Discussion
sure (CMDM) database [7]. The database is generated from 1) The Performance of PCQA Validated on the SJTU-PCQA
5 source models subjected to geometry and color distortions. and WPC Databases: The experimental results for PCQA on
Then the source models are corrupted with 4 types of distor- the SJTU-PCQA database and the WPC database are shown
tions based on color and geometry and each type of distortion in Table II. The best performance results of the NR-PCQA
is adjusted with 4 different strengths. The Ai x, Chamleon, methods along with the best performance results of all methods
Fi sh, and Samur ai models are selected for training while the are marked in bold. We can see clearly that the FR-PCQA
Ari model is chosen for testing. Specifically, each distorted metric PCQM outperforms other PCQA metrics. With the
model is provided with 5 subjective scores according to its assistance of reference information, FR-PCQA methods tend
viewpoints and animation types. For simplification, we use the to be more effective at predicting the quality levels of colored
average of the 5 subjective scores as the final quality score for point clouds than NR-PCQA methods. The proposed method
the distorted model. achieves first place on the SJTU-PCQA database and second
In the literature, few metrics are proposed to deal with the place on the WPC database among the compared NR-PCQA
3D-QA tasks of color meshes. In order to evaluate the per- metrics respectively. It’s worth mentioning that the PQA-Net
formance of the proposed method, some image-based metrics extracts features from the rendering views with deep-learning
that might be able to predict the visual quality of 3D meshes networks. The proposed method depends on handcrafted fea-
are utilized as competitors. Unfortunately, few no-reference tures and remains competitive considering the computational
quality assessment metrics for 3D meshes are open-sourced, resource consumption of deep-learning networks.
thus we try to reproduce some of the metrics. Specifically, the With closer observations, we can make several more detailed
metrics used for comparison can be divided into two types: analyses: 1-1) Image-based metrics using simple IQA models
• Image-based metrics: These metrics operate by evaluating (such as PSNR and SSIM) are less effective. This is because
the quality of 2D images rendered from the 3D colored the rendering views captured from various viewpoints are quite
meshes. The FR image-based metrics include PSNR different in content. To get more quality information, more
and SSIM [26]. The NR image-based metrics include snapshots have to be sampled, which may confuse such IQA
NIQE [52] and BRISQUE [38]. models because they are not capable of dealing with multiple
• Model-based metrics: The FR-MQA metric includes pairs of images with dissimilar contents. Thanks to the strong
CMDM [7]. The NR-MQA metrics designed especially learning ability of CNN, PQA-Net overcomes the difficulty

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 ZHANG et al.: NR QUALITY ASSESSMENT FOR 3D COLORED POINT CLOUD AND MESH MODELS 7627

Fig. 5. Illustration of point clouds with consistently increasing downsampling distortion levels from the SJTU-PCQA [6] and the WPC [10] databases. (a),
(b), and (c) are the snapshots of rednandblack_15, rednandblack_16, and rednandblack_17 from the SJTU-PCQA database. (d), (e), and (f) are the snapshots
of banana_level_9, banana_level_8, and banana_level_7 from the WPC database respectively.

TABLE III
P ERFORMANCE C OMPARISON W ITH C OMPETITORS
ON THE CMDM D ATABASE

Fig. 6. Examples of meshes with increasing geometry quantization dis-

tortion levels from the CMDM database [7]. (a), (b), (c), and (d) are the
and achieves high performance by utilizing the correlation of snapshots of Ari_QuantGeom_1, Ari_QuantGeom_2, Ari_QuantGeom_3, and
the rendering views. To conclude, we think that increasing the Ari_QuantGeom_4 respectively.
sample number of snapshots and optimizing learning models
are the focus of image-based methods in the future. However, of all methods are marked in bold. Surprisingly, the FR
such methods inevitably require large amounts of computation image-based metrics get remarkable performance results. It is
resources and highly depend on the scale of the database. because the CMDM database manually adjusts the employed
1-2) Most model-based methods achieve lower performance 4 strengths of distortions to cover most range of visual
on the WPC database than on the SJTU-PCQA database. quality levels (from imperceptible levels to high levels of
We attempt to give the reasons. First, although the two impairment), which makes the distortion levels more coarse-
databases manually introduce similar types of distortions, the grained. As illustrated in Fig. 6, the degradations caused by
WPC database employs some point cloud samples that are the 4 distortion strengths are quite obvious, thus enabling such
less sensitive to distortions in perceived visual quality. For simple methods to gain competitive performance.
example, as can be seen in Fig. 5, the rednandblack sample The NR model-based MQA methods such as NR-SVR, NR-
from the SJTU-PCQA database appears to be more distinct GRNN, and NR-CNN have a stronger ability to predict the
under the influence of downsampling distortions. However, visual quality than the NR image-based MQA methods, but
the banana sample from the WPC database is less complex they do not take color information into consideration, thus
in geometry structure and limited in colorfulness, which resulting in lower performance than the proposed method.
makes the distortion difference less obvious. Second, the The FR-MQA metric CMDM achieves first place in PLCC
SJTU-PCQA database includes distorted point clouds with and SRCC and the proposed method obtains second place
mixed distortions while the WPC database only adds one type in PLCC and SRCC with a slight gap. The reason is that
of distortion to a single point cloud. It seems that point clouds the coarse-grained levels of distortions make it easier for the
with mixed distortions are more distinguishable in quality proposed method to learn the relationship between the corre-
under close distortion levels. What’s more, the WPC database sponding 3D-NSS parameters and quality scores. Therefore,
has twice as many reference point clouds as the SJTU-PCQA even with relatively little training data, the proposed method
database. The expanded scale and content diversity challenge still obtains good performance.
the generalization ability of such methods as well.
2) The Performance of MQA Validated on the CMDM
Database: Table III presents the experimental results on the E. Ablation Study
CMDM database, in which the top performance results of the To further test the effectiveness and contributions of dif-
NR-PCQA methods along with the best performance results ferent types of features (color and geometry) and different

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 7628 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 32, NO. 11, NOVEMBER 2022

TABLE IV
P ERFORMANCE OF THE A BLATION S TUDY. T HE B EST P ERFORMANCE FOR E ACH D ATABASE I S M ARKED IN B OLD

statistical parameters, we split the features into 8 groups: quality score on all three databases. It might be because all
(1) F1: mean, standard deviation, and entropy values of geome- three databases introduce more types of geometry distortions
try feature domains; (2) F2: GGD parameters of geometry fea- than color distortions and the geometry information weighs
ture domains; (3) F3: AGGD parameters of geometry feature more in the human perception of 3D models.
domains; (4) F4: Gamma distribution parameters of geometry By analyzing the different performances of different models,
feature domains; (5) F5: mean, standard deviation, and entropy we can make several observations. 1) On the SJTU-PCQA
values of color feature domains; (6) F6: GGD parameters of database, we can see that the mean, standard deviation, and
color feature domains; (7) F7: AGGD parameters of color entropy values obtain the best performance, even higher than
feature domains; (8) F8: Gamma distribution parameters of the proposed all groups. While on the WPC database, the all
color feature domains. Then we can analyze the features’ groups model is significantly superior to the separate distrib-
contributions by obtaining the performance of different combi- ution models. We attempt to give our explanation as follows.
nations of feature groups. For example, the F2+F6 model only The ability of simple parameters to describe the shape of
contains AGGD parameters of both geometry and color feature distributions is limited. For example, some distributions with
domains while the F1˜F4 model only consists of statistical very different shapes may have the same mean and variance.
parameters of geometry feature domains. When the diversity and number of samples are not saturated,
Besides, most 3D models are perceived by humans only simpler parameters may achieve better results (more complex
through the rendered 2D views. To find out whether the parameters are troubled by the redundant information), but
information of 2D views would help improve the perfor- when the diversity and number of samples increase, these
mance of 3D-NSS, we test the effectiveness of the combined simpler parameters are less effective, which requires the help
2D-NSS and 3D-NSS features as well. The 2D-NSS features of more complex parameters to describe the distributions.
are extracted by the 2D views of objects using BRISQUE [38] Therefore, we suggest using the proposed all groups model
and NIQE [52] (The redundant features of BRISQUE and for practical application since such model is more capa-
NIQE are removed). The features extracted from different ble of describing and distinguishing different distributions.
viewpoints are averaged to get the final 2D-NSS features. The 2) It seems that the 2D-NSS is not effective for 3D-QA tasks.
proposed all groups’ features in this paper are employed as We try to give the reason for such phenomenon as well. Single
the 3D-NSS features. We simply concatenate the 2D-NSS and rendering 2D views contain limited information, therefore,
3D-NSS features as the quality-aware feature vector. we have to use multiple viewpoints to cover more quality
The performance results of the ablation study are shown information. However, the contents of snapshots from various
in Table IV, where the mean, standard deviation, and entropy viewpoints are quite different, such as the top and bottom
values of geometry and color feature domains are represented views of the 3D models. The contents difference may have
as the basic type of features (F1 + F5). It can be obviously a much greater influence on the distributions rather than the
observed that the geometry features contribute more to the final distortions.

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 ZHANG et al.: NR QUALITY ASSESSMENT FOR 3D COLORED POINT CLOUD AND MESH MODELS 7629

Fig. 7. Statistical test results of the proposed method and compared metrics on the SJTU-PCQA, WPC, and CMDM databases. A black/white block means
the row method is statistically worse/better than the column one. A gray block means the row method and the column method are statistically indistinguishable.
The metrics are denoted by the same index as in Table II and Table III respectively.

TABLE V
P ERFORMANCE OF THE D ATA -S ENSITIVITY E XPERIMENT. T HE B EST P ERFORMANCE FOR E ACH D ATABASE I S M ARKED IN B OLD

F. Statistical Test
To further analyze the performance of the proposed method,
we conduct the statistical test in this section. We follow the
same experiment setup as in [56] and compare the difference
between the predicted quality scores with the subjective rat-
ings. All possible pairs of models are tested and the results
are listed in Fig. 7. It can be seen that our method is
significantly superior to 7 compared PCQA metrics on the
SJTU-PCQA and the WPC databases while our method also
significantly outperforms 7 compared MQA metrics on the
CMDM database. Specifically, the FR metric PCQM achieves
significantly better performance than our method on both
SJTU-PCQA and WPC databases, the FR metric CMDM is
insignificantly distinguishable from our method on the CMDM
database. Fig. 8. The results of time cost comparison on the SJTU-PCQA and the
WPC databases. The time cost refers to the average time consumption per
point cloud for each database.
G. Data-Sensitivity Experiment
To find out the influence of the number of training samples, the performance more significantly. For the PCQA database,
we conduct the data-sensitivity experiment by changing the we think using fewer training samples may exaggerate the
proportion of training set as about 20%, 40%, 60%, and 80% impact of noisy labels and result in the over-fit of the model,
(2, 4, 6, 8 sample groups for training on the SJTU-PCQA which causes the performance to drop. In all, increasing the
database, 4, 8, 12, 16 for training on the WPC database, and 1, diversity and number of the training samples are beneficial for
2, 3, 4 samples for training on the CMDM database corre- obtaining more robustness and higher performance.
spondingly). The experimental results are exhibited in Table V,
from which we can find that increasing the number of training
samples is helpful to improve the performance of the pro- H. Computational Efficiency
posed method. The CMDM database employs coarse-grained Considering that the proposed method operates directly
distortion levels and includes relatively small amounts of from the 3D model, we also focus on computational efficiency.
mesh samples, thus the increase of training data improves The image-based metrics usually operate quite fast due to

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 7630 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 32, NO. 11, NOVEMBER 2022

the mature development of IQA metrics. Therefore, to make [10] H. Su, Q. Liu, Z. Duanmu, W. Liu, and Z. Wang, “Perceptual quality
the comparison meaningful, we select model-based methods assessment of colored 3D point clouds,” 2021, arXiv:2111.05474.
[11] W. Sun, X. Min, G. Zhai, and S. Ma, “Blind quality assessment for
GraphSIM, PointSSIM, PCQM, and PCMRR as competitors. in-the-wild images via hierarchical feature fusion and iterative mixed
We conduct the test on a computer with Intel (R) Core (TM) database training,” 2021, arXiv:2105.14550.
i5-3470 CPU @ 3.20 GHz and 8 GB RAM on the Windows [12] P. Cignoni, C. Rocchini, and R. Scopigno, “METRO: Measuring error on
platform. The corresponding time cost results are shown in simplified surfaces,” Comput. Graph. Forum, vol. 17, no. 2, pp. 167–174,
Sep. 1998.
Fig. 8, and we can see clearly that the average time cost of the
[13] R. Mekuria and P. Cesar, “MP3DG-PCC, open source software frame-
proposed method is smaller than all the compared model-based work for implementation and evaluation of point cloud compression,”
methods, which indicates that our method achieves relatively in Proc. 24th ACM Int. Conf. Multimedia, 2016, pp. 1222–1226.
considerable computational efficiency. [14] G. Lavoué, “A multiscale metric for 3D mesh visual quality assessment,”
Comput. Graph. Forum, vol. 30, no. 5, pp. 1427–1437, 2011.
[15] K. Wang, F. Torkhani, and A. Montanvert, “A fast roughness-based
V. C ONCLUSION approach to the assessment of 3D mesh visual quality,” Comput. Graph.,
vol. 36, no. 7, pp. 808–818, Nov. 2012.
This paper proposes a no-reference colored 3D model
[16] D. Tian, H. Ochimizu, C. Feng, R. Cohen, and A. Vetro, “Geometric
quality assessment metric based on entropy and 3D natural distortion metrics for point cloud compression,” in Proc. IEEE Int. Conf.
scene statistics. The proposed method deals with the quality Image Process. (ICIP), Sep. 2017, pp. 3460–3464.
assessment problems for both colored point cloud and mesh [17] E. Alexiou and T. Ebrahimi, “Point cloud quality assessment metric
models. We first project the 3D models into corresponding based on angular similarity,” in Proc. IEEE Int. Conf. Multimedia Expo
(ICME), Jul. 2018, pp. 1–6.
quality-related geometry and color feature domains. Then [18] A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “A generalized
entropy and various 3D-NSS parameters are estimated to Hausdorff distance based quality metric for point cloud geometry,” in
better capture the representative characteristics and quan- Proc. 12th Int. Conf. Quality Multimedia Exper. (QoMEX), May 2020,
pp. 1–6.
tify the distortions that are more in line with human per-
[19] L. Váša and J. Rus, “Dihedral angle mesh error: A fast per-
ception. The proposed method is validated on the colored ception correlated distortion measure for fixed connectivity triangle
point cloud quality assessment database (SJTU-PCQA), the meshes,” Comput. Graph. Forum, vol. 31, no. 5, pp. 1715–1724,
Waterloo point cloud assessment database (WPC), and the Aug. 2012.
colored mesh quality assessment database (CMDM). The exp- [20] E. Alexiou and T. Ebrahimi, “Towards a point cloud structural similarity
metric,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops (ICMEW),
erimental results show that our method outperforms most Jul. 2020, pp. 1–6.
compared NR 3D-QA metrics with competitive computational [21] D. Tian and G. AlRegib, “BaTex3: Bit allocation for progressive
resources and reduces the performance gap with the state- transmission of textured 3-D models,” IEEE Trans. Circuits Syst. Video
of-the-art FR 3D-QA metrics. The proposed method follows Technol., vol. 18, no. 1, pp. 23–35, Jan. 2008.
[22] J. Guo, V. Vidal, I. Cheng, A. Basu, A. Baskurt, and G. Lavoue,
a common NR framework and can be easily modified and “Subjective and objective visual quality assessment of textured 3D
expanded to satisfy specific needs, which has great application meshes,” ACM Trans. Appl. Perception, vol. 14, no. 2, pp. 1–20,
potential. The code of the proposed method is released for Feb. 2017.
promoting the development of NR 3D-QA. [23] G. Meynet, Y. Nehmé, J. Digne, and G. Lavoué, “PCQM: A full-
reference quality metric for colored 3D point clouds,” in Proc.
12th Int. Conf. Quality Multimedia Exper. (QoMEX), May 2020,
R EFERENCES pp. 1–6.
[24] Q. Liu, H. Yuan, R. Hamzaoui, H. Su, J. Hou, and H. Yang, “Reduced
[1] H. Graf, S. P. Serna, and A. Stork, “Adaptive quality meshing for ‘on- reference perceptual quality model with application to rate control for
the-fly’ volumetric mesh manipulations within virtual environments,” video-based point cloud compression,” IEEE Trans. Image Process.,
in Proc. IEEE Symp. Virtual Environ., Hum.-Comput. Interfaces Meas. vol. 30, pp. 6623–6636, 2021.
Syst., Jul. 2006, pp. 178–183. [25] Z. Zhang et al., “A no-reference visual quality metric for 3D color
[2] L. Li, Z. Li, S. Liu, and H. Li, “Occupancy-map-based rate distortion meshes,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops
optimization and partition for video-based point cloud compression,” (ICMEW), Jul. 2021, pp. 1–6.
IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 1, pp. 326–338,
[26] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image
Jan. 2021.
quality assessment: From error visibility to structural similarity,” IEEE
[3] R. Mekuria, K. Blom, and P. Cesar, “Design, implementation, and
Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.
evaluation of a point cloud codec for tele-immersive video,” IEEE Trans.
Circuits Syst. Video Technol., vol. 27, no. 4, pp. 828–842, Apr. 2017. [27] Y. Liu, Q. Yang, Y. Xu, and L. Yang, “Point cloud quality assessment:
Dataset construction and learning-based no-reference approach,” 2020,
[4] (May 1, 2022). Apple Developer Document: Overview of
arXiv:2012.11895.
Scanning and Detecting 3-Dimension Objects. [Online]. Avail-
able: https://wrywhisker.pulpfriction.net/wallcrust/linear-colinear- [28] R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas, “PointNet:
felinear.html Deep learning on point sets for 3D classification and segmentation,”
[5] (May 1, 2022). Intel Realsense. [Online]. Available: https://www. in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017,
intelrealsense.com/ pp. 652–660.
[6] Q. Yang, H. Chen, Z. Ma, Y. Xu, R. Tang, and J. Sun, “Predicting [29] Q. Liu et al., “PQA-Net: Deep, no., reference point cloud quality
the perceptual quality of point cloud: A 3D-to-2D projection-based assessment via multi-view projection,” IEEE Trans. Circuits Syst. Video
exploration,” IEEE Trans. Multimedia, vol. 23, pp. 3877–3891, 2021. Technol., vol. 31, no. 12, pp. 4645–4660, Dec. 2021.
[7] Y. Nehmé, F. Dupont, J. P. Farrugia, P. Le Callet, and G. Lavoué, “Visual [30] I. Abouelaziz, M. El Hassouni, and H. Cherifi, “No-reference 3D
quality of 3D meshes with diffuse colors in virtual reality: Subjective mesh quality assessment based on dihedral angles model and support
and objective evaluation,” IEEE Trans. Vis. Comput. Graphics, vol. 27, vector regression,” in Proc. Int. Conf. Image Signal Process. Cham,
no. 3, pp. 2202–2219, Mar. 2021. Switzerland: Springer, May 2016, pp. 369–377.
[8] E. Alexiou, I. Viola, T. M. Borges, T. A. Fonseca, R. L. de Queiroz, and [31] I. Abouelaziz, M. E. Hassouni, and H. Cherifi, “A convolutional neural
T. Ebrahimi, “A comprehensive study of the rate-distortion performance network framework for blind mesh visual quality assessment,” in Proc.
in MPEG point cloud compression,” APSIPA Trans. Signal Inf. Process., IEEE Int. Conf. Image Process. (ICIP), Sep. 2017, pp. 755–759.
vol. 8, no. 1, p. e27, 2019. [32] I. Abouelaziz, A. Chetouani, M. El Hassouni, L. J. Latecki, and
[9] E. M. Torlig, E. Alexiou, T. A. Fonseca, R. L. de Queiroz, and H. Cherifi, “No-reference mesh visual quality assessment via ensemble
T. Ebrahimi, “A novel methodology for quality assessment of voxelized of convolutional neural networks and compact multi-linear pooling,”
point clouds,” Proc. SPIE, vol. 10752, pp. 174–190, Sep. 2018. Pattern Recognit., vol. 100, Apr. 2020, Art. no. 107174.

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.
 ZHANG et al.: NR QUALITY ASSESSMENT FOR 3D COLORED POINT CLOUD AND MESH MODELS 7631

[33] S. Yang, C.-H. Lee, and C.-C.-J. Kuo, “Optimized mesh and texture Zicheng Zhang (Student Member, IEEE) received
multiplexing for progressive textured model transmission,” in Proc. 12th the B.E. degree from Shanghai Jiao Tong University,
Annu. ACM Int. Conf. Multimedia, 2004, pp. 676–683. Shanghai, China, in 2020, where he is currently
[34] F. Caillaud, V. Vidal, F. Dupont, and G. Lavoué, “Progressive com- pursuing the Ph.D. degree with the School of Elec-
pression of arbitrary textured meshes,” Comput. Graph. Forum, vol. 35, tronic Information and Electrical Engineering. His
no. 7, pp. 475–484, Oct. 2016. research interests include image quality assessment,
[35] Z. Wang, E. Simoncelli, and A. Bovik, “Multiscale structural similarity video quality assessment, and 3D visual quality
for image quality assessment,” in Proc. Asilomar Conf. Signals, Syst. assessment.
Comput., vol. 2, 2003, pp. 1398–1402.
[36] L. Zhang, L. Zhang, X. Mou, and D. Zhang, “FSIM: A feature similarity
index for image quality assessment,” IEEE Trans. Image Process.,
vol. 20, no. 8, pp. 2378–2386, Aug. 2011. Wei Sun received the B.E. degree from the
[37] H. R. Sheikh and A. C. Bovik, “Image information and visual quality,” East China University of Science and Technology,
IEEE Trans. Image Process., vol. 15, no. 2, pp. 430–444, Feb. 2006. Shanghai, China, in 2016. He is currently pursu-
[38] A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality ing the Ph.D. degree with the Institute of Image
assessment in the spatial domain,” IEEE Trans. Image Process., vol. 21, Communication and Network Engineering, Shanghai
no. 12, pp. 4695–4708, Dec. 2012. Jiao Tong University. His research interests include
[39] A. K. Moorthy and A. C. Bovik, “Blind image quality assessment: image quality assessment, perceptual signal process-
From natural scene statistics to perceptual quality,” IEEE Trans. Image ing, and mobile video processing.
Process., vol. 20, no. 12, pp. 3350–3364, Dec. 2011.
[40] Y. Lin, M. Yu, K. Chen, G. Jiang, Z. Peng, and F. Chen, “Blind mesh
quality assessment method based on concave, convex and structural
features analyses,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops Xiongkuo Min (Member, IEEE) received the B.E.
(ICMEW), Jul. 2019, pp. 282–287. degree from Wuhan University, Wuhan, China,
[41] I. Abouelaziz, M. El Hassouni, and H. Cherifi, “Blind 3D mesh visual in 2013, and the Ph.D. degree from Shanghai
quality assessment using support vector regression,” Multimedia Tools Jiao Tong University, Shanghai, China, in 2018.
Appl., vol. 77, no. 18, pp. 24365–24386, Sep. 2018. From January 2016 to January 2017, he was a
[42] Q. Mérigot, M. Ovsjanikov, and L. J. Guibas, “Voronoi-based curvature Visiting Student at the Department of Electrical
and feature estimation from point clouds,” IEEE Trans. Vis. Comput. and Computer Engineering, University of Waterloo,
Graph., vol. 17, no. 6, pp. 743–756, Jun. 2011. Canada. He is currently a Post-Doctoral Fellow
[43] H. Thomas, J. Deschaud, B. Marcotegui, F. Goulette, and Y. L. Gall, with Shanghai Jiao Tong University. His research
“Semantic classification of 3D point clouds with multiscale spherical interests include visual quality assessment, visual
neighborhoods,” 2018, arXiv:1808.00495. attention modeling, and perceptual signal process-
[44] T. Hackel, J. D. Wegner, and K. Schindler, “Fast semantic segmen- ing. He received the Best Student Paper Award from IEEE ICME in 2016.
tation of 3D point clouds with strongly varying density,” ISPRS Ann.
Photogramm., Remote Sens. Spatial Inf. Sci., vol. 3 pp. 177–184, Tao Wang received the B.E. degree from Shanghai
Jul. 2016. Jiao Tong University, Shanghai, China, in 2020,
[45] T. Surazhsky, E. Magid, O. Soldea, G. Elber, and E. Rivlin, “A compar- where he is currently pursuing the M.S. degree
ison of Gaussian and mean curvatures estimation methods on triangular with the School of Electronic Information and Elec-
meshes,” in Proc. IEEE Int. Conf. Robot. Autom., vol. 1, Sep. 2003, trical Engineering. His research interests include
pp. 1021–1026. image/video quality assessment and image aesthetic
[46] P. Alliez, D. Cohen-Steiner, O. Devillers, B. Lévy, and M. Desbrun, quality assessment.
“Anisotropic polygonal remeshing,” in Proc. ACM SIGGRAPH Papers,
2003, pp. 485–493.
[47] M. Corsini, E. D. Gelasca, T. Ebrahimi, and M. Barni, “Watermarked
3-D mesh quality assessment,” IEEE Trans. Multimedia, vol. 9, no. 2,
pp. 247–256, Feb. 2007. Wei Lu received the B.E. degree from Shanghai
[48] N. Mukherjee, “A hybrid, variational 3D smoother for orphaned shell Jiao Tong University, Shanghai, China, in 2020,
meshes,” in Proc. IMR, 2002, pp. 1–12. where he is currently pursuing the master’s degree
[49] H. Lee, P. Alliez, and M. Desbrun, “Angle-analyzer: A triangle-quad with the Department of Electronic Engineering. His
mesh codec,” Comput. Graph. Forum, vol. 21, no. 3, pp. 383–392, research interests include image quality assessment
Sep. 2002. and video quality assessment.
[50] K. Sharifi and A. Leon-Garcia, “Estimation of shape parameter for
generalized Gaussian distributions in subband decompositions of video,”
IEEE Trans. Circuits Syst. Video Technol., vol. 5, no. 1, pp. 52–56,
Feb. 1995.
[51] (May 1, 2022). Scikit-Learn for Python. [Online]. Available:
https://scikit-learn.org/stable/ Guangtao Zhai (Senior Member, IEEE) received
[52] A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a ‘completely the B.E. and M.E. degrees from Shandong Uni-
blind’ image quality analyzer,” IEEE Signal Process. Lett., vol. 20, no. 3, versity, Shandong, China, in 2001 and 2004,
pp. 209–212, Nov. 2013. respectively, and the Ph.D. degree from Shanghai
[53] Q. Yang, Z. Ma, Y. Xu, Z. Li, and J. Sun, “Inferring point cloud quality Jiao Tong University, Shanghai, China, in 2009.
via graph similarity,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, From 2008 to 2009, he was a Visiting Student
no. 6, pp. 3015–3029, Jun. 2022. with the Department of Electrical and Computer
[54] I. Viola and P. Cesar, “A reduced reference metric for visual quality Engineering, McMaster University, Hamilton, ON,
evaluation of point cloud contents,” IEEE Signal Process. Lett., vol. 27, Canada, where he was a Post-Doctoral Fellow
pp. 1660–1664, 2020. from 2010 to 2012. From 2012 to 2013, he was
[55] I. Abouelaziz, M. El Hassouni, and H. Cherifi, “A curvature based a Humboldt Research Fellow with the Institute of
method for blind mesh visual quality assessment using a general regres- Multimedia Communication and Signal Processing, Friedrich Alexander Uni-
sion neural network,” in Proc. 12th Int. Conf. Signal-Image Technol. versity of Erlangen–Nuremberg, Germany. He is currently a Research Profes-
Internet-Based Syst. (SITIS), 2016, pp. 793–797. sor with the Institute of Image Communication and Information Processing,
[56] H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical eval- Shanghai Jiao Tong University. His research interests include multimedia
uation of recent full reference image quality assessment algo- signal processing and perceptual signal processing. He received the Award
rithms,” IEEE Trans. Image Process., vol. 15, no. 11, pp. 3440–3451, of National Excellent Ph.D. Thesis from the Ministry of Education of China
Nov. 2006. in 2012.

Authorized licensed use limited to: University Roma Tre AREA SCIENTIFICO TECNOLOGICA. Downloaded on June 26,2023 at 06:02:19 UTC from IEEE Xplore. Restrictions apply.