BOSS: Bones, Organs and Skin Shape Model

Karthik Shetty1,2 Annette Birkhold2 Srikrishna Jaganathan1,2 Norbert Strobel2 Bernhard Egger1
Markus Kowarschik2 Andreas Maier1
1 2
FAU Erlangen-Nürnberg, Erlangen, Germany Siemens Healthineers AG, Forchheim, Germany
[email protected]
arXiv:2303.04923v1 [cs.CV] 8 Mar 2023

Abstract information about a patient is always available, a statistical

model of body shape can serve as a foundation for incorpo-
Objective: A digital twin of a patient can be a valuable rating patient-specific information. Such a Statistical Shape
tool for enhancing clinical tasks such as workflow automa- Model (SSM) of the human anatomy typically represents
tion, patient-specific X-ray dose optimization, markerless the average shape of multiple subjects and their variation
tracking, positioning, and navigation assistance in image- in shape using a low-dimensional parameter space [1, 2].
guided interventions. However, it is crucial that the pa- Further, by incorporating kinematics, thereby the pose, the
tient’s surface and internal organs are of high quality for model can realistically represent a human body during a
any pose and shape estimates. At present, the majority of medical intervention.
statistical shape models (SSMs) are restricted to a small The availability of such patient models opens up pos-
number of organs or bones or do not adequately represent sibilities for automating clinical workflow steps entirely
the general population. Method: To address this, we pro- based on simulations. For example, in interventional en-
pose a deformable human shape and pose model that com- vironments such a model can be used to estimate and re-
bines skin, internal organs, and bones, learned from CT duce the X-ray dose distribution within the patient as well
images. By modeling the statistical variations in a pose- as the dose the staff is exposed to due to scattered radia-
normalized space using probabilistic PCA while also pre- tion [3,4]. Other applications are the automated positioning
serving joint kinematics, our approach offers a holistic rep- of the imaging system relative to a target region, such as an
resentation of the body that can benefit various medical ap- organ, or the virtual display of X-ray images that would be
plications. Results: We assessed our model’s performance acquired based on the current system settings and position
on a registered dataset, utilizing the unified shape space, and patient pose prior to any X-ray exposure. It was also
and noted an average error of 3.6 mm for bones and 8.8 demonstrated such a generation of virtual X-ray images for
mm for organs. To further verify our findings, we con- bony structures based on pre-existing 3D hip models which
ducted additional tests on publicly available datasets with are later iteratively fit to a patient’s X-ray [5, 6].
multi-part segmentations, which confirmed the effectiveness
of our model. Conclusion: This works shows that anatomi- A virtual twin of the patient could be further employed
cally parameterized statistical shape models can be created for enhanced segmentation of anatomical structures [7], if
accurately and in a computationally efficient manner. Sig- (preoperative) images of a patient are available or are gen-
nificance: The proposed approach enables the construction erated during the intervention. This would further facilitate
of shape models that can be directly applied to various med- the (pre- or intraoperative) generation of a patient-specific
ical applications, including biomechanics and reconstruc- hierarchical model. It has been shown, that coupling a sta-
tion. tistical knee model with a segmentation neural network a
more precise segmentation of the knee based on magnetic
resonance images can be achieved [8]. Shape models can
1. Introduction be beneficial for the segmentation of 3D volumetric im-
ages such as Computed Tomography (CT) or Magnetic Res-
A virtual patient model that incorporates anatomy and onance Imaging (MRI) [9–11], as well as for 2D projec-
physiology has the potential to enhance numerous medi- tion images such as X-rays [12]. This is because the prior
cal diagnosis and therapy tasks. In particular, in the field knowledge of the body shape can be used as a regularizer.
of minimally invasive image-guided procedures, a detailed Employed on a full body patient model, this technique could
understanding of the patient being treated could facilitate be applied online during image-guided (MR or X-ray) med-
(semi-) automated treatments. Given that not all necessary ical interventions and might therefore facilitate navigation
during the procedure. Additionally, shape models can be hance the model’s representation, it was further extended to
used to generate synthetic data to train deep neural net- handle CT scans from individuals of varying age, height,
works, allowing them to better generalize [13, 14] to new and body mass [22, 23]. However, this process is compu-
subjects and overcome the limited anatomical variation of- tationally expensive since each segmented CT scan must
ten observed during domain translation. be registered to a template, followed by large deformation
Anatomic body models are similar to statistical shape diffeomorphic metric mapping between the initial XCAT
models in that they are both computer-generated models model and the registered scan. Furthermore, the robust-
of the human anatomy. However, anatomic body models ness of this method decreases when patients are in different
are designed to provide a highly detailed representation of poses.
the anatomy of a single individual, while statistical shape Various techniques have been introduced to create realis-
models aim to capture the variation in shape that occurs tic models of human skin for individuals using captured 3D
across the population. Anatomic body models typically range scans that modify a generic 3D skin template to fit the
include different body parts, such as bones, muscles, or- particular person. For instance, Allen et al. employed non-
gans, and vessels [15, 16]. They can be used for a variety rigid template fitting to compute correspondences between
of purposes, including visualization of anatomy for educa- human body shapes in similar poses [24]. This method has
tional purposes, understanding relationships between differ- been extended to work for varying poses [25, 26]. A dataset
ent structures, and simulations of medical procedures. De- of varying shapes and poses can be utilized to create a data-
spite their level of detail, the focus on a single individual’s driven model for building a human skin surface shape model
anatomy may limit their practicality in real-world applica- that incorporates these variations [1, 2, 27]. These mod-
tions. We hold the view that by leveraging the abundance els represent the variations in human shape identity using a
of data and the progress in deep learning, the gap between PCA space, and variations due to pose using an articulated
statistical models and anatomical models can be bridged skeleton-based skinning approach.
through a data-driven approach. As these methods have shown promising results for the
While prior studies in shape modeling focused primarily human skin, in this paper we extend the existing method
on specific anatomical regions (few bones or organs), this based on SMPL [1] to combine skin, skeleton and internal
research aims to develop a comprehensive model represent- organs as one parameterized full body model. We start by
ing the entire human body. Numerous works have proposed creating an articulated template mesh for skeleton and or-
various solutions to create personalized anatomical mod- gans. Followed by registration of the individual structures,
els, such as Meng et.al.’s realistic spine model, which was which is then pose-normalized to a rest pose. The shape
learned from partial scans [17]. Kadlecek et.al. proposed space is then learnt on the rest pose. This is the first joint
personalized anatomical models for physics-based anima- model for bones, organs and skin covering the full body.
tion using only the surface 3D scans of skin [18]. However, This demonstrates the applicability of the concepts of statis-
the bone structures in these models are based on uniform tical shape models to a full-body scale, encompassing mul-
scaling, which may not accurately represent the true struc- tiple layers. Such a comprehensive model enables holistic
ture. While the individual components of the BASH model image processing and we expect it to be highly valuable to
provide accurate representations of kinematics and skin sur- improve image acquisition and understanding.
face, combining them using linear interpolation may not
fully capture the true anatomy. The most similar model to
2. Methodology
our work is likely the recent OSSO model, which combines Our statistical model is based on a set publicly available
a skin and bone model [19]. OSSO is a data-driven model whole body as well as partial CT scans (total ≈ 300). We
that infers the shape of the skeleton from a given human sur- performed automatic segmentation on them to isolate skele-
face scan. However, it is important to note that the skeleton tons, internal organs and skin surfaces to model those com-
model in OSSO is learned solely from 2D data, even though ponents individually. Unlike SMPL or other well known
the method is data-driven. statistical parametric models, the availability of a pose-only
In the field of patient modelling, the most prominent dataset is not feasible as we cannot subject a patient with
approach describing whole body anatomies is based on unnecessary radiation. Usually while modelling the human
4D extended cardiac-torso (XCAT) [20] employing nonuni- skin, the pose-dataset is primarily used to learn the pose re-
form rational B-spline (NURBS). The underlying model is lated shape variations, such as the bulging of the muscles.
based on male and female CT scan from the Visible Hu-
2.1. SMPL Introduction
man dataset [21] which were manually segmented on a
fixed pose. The model was extended to incorporate cardiac We provide necessary background on the SMPL frame-
and respiratory motions as a parametric function, based on work, which forms the basis of our work [1]. The SMPL
cardiac- and respiratory-gated multislice CT data. To en- model is a statistical parametric function M (β, θ, t; Φ),
where β are the shape parameters, θ are the pose param- Here, wk,i is an element of the blend weights W that corre-
eters, t represents the global translation and, Φ represents sponds to segment k and vertex index i, and Gk =[Gk |q k ] is
the learned model parameters. The output of this func- the global transformation of joint k.
tion is a triangulated surface mesh with N = 6890 ver-
tices. The shape parameters β are represented as low- 2.2. Data
dimensional PCA coefficients, learned from a standard In this study, 3D CT images of nearly 300 patients were
shape dataset [28] with BS (β) : R|β| 7→ R3N represent- used for the statistical modeling. Due to the limited pub-
ing offsets to a mean template mesh T̄ in the rest pose. lic availability of whole body CT scans, we relied on two
The pose of the model is defined by a kinemat- types of datasets, whole-body scans and partial-body scans.
ics chain with a set of relative rotation matrices R = Whole-body scans typically consists of the entire human
[R1 , . . . , RK ] ∈ R3×3 made up of K = 24 joints. Each body from head to toe with a exceptions missing parts
set of rotation matrices is a function of the pose parame- around the arm regions. In total, 42 valid scans from the
ters θ i ∈ R3 which represents the axis-angle rotations rel- Visceral dataset [30] make up the whole-body scans with
ative to the joints. Deforming the template model from its an average voxel resolution of 0.87 × 0.87 × 2.0 mm. The
rest pose to a desired pose based on the relative rotations partial-body scans, on the other hand, predominantly cover
is known as forward kinematics. Let tk ∈ R3 represent scan areas ranging from the neck to the femur region. This
the joint locations of the rest pose template. The new joint set comprises 58 valid scans from the Visceral dataset [30],
locations q k ∈ R3 of the deformed model are determined and 206 valid scans from the QIN-HeadNeck dataset [31]
by with an average voxel resolution of 0.91 × 0.91 × 1.5 mm
and 0.97×0.97×2.0 mm, respectively. Additional metadata
q k = Gk (tk − tp(k) ) + q p(k) , such as the height and weight of the patient during the CT
acquisition was available for the QIN-HeadNeck dataset.
where Gk =Rk Gp(k) ∈ R3×3 are the global rotation of On both datasets we have only included patients who have
each joint k calculated recursively based on the kinematic 12 thoracic vertebra and 5 lumbar vertebra as we are inter-
tree. Here, p(k) represents the parent joint for joint k. ested in the general population who can modeled based off
For more in depth description and implementations refer the template skeleton model. Hence, we cannot account for
to [1, 29]. varying amount of bones. In addition we excluded all pa-
The deformation process requires additional parameters tients with significant metallic implants as they cause imag-
such as the joint regressor J (.), blend weights W and ing artifacts which degrade the quality of the segmented CT
pose-dependent shape variations BP (.). The joint regres- images. We further rejected patients with missing organs
sor J (β) computes the new joint location tk for differ- such as the kidneys. In total, we make use of 306 CT scans
ent body shapes in the rest pose. The blend weights and for building the model.
pose-dependent shape variations are learned from a pose We anatomically segmented the 3D scans into 3 distinct
dataset consisting of multiple subjects in various poses. The sets, namely skin, bones and organs. This was automati-
smoothing function of rotating vertices around a given joint cally performed using the AI-Rad Companion Organs RT1
is determined by the blend weights W, using a given blend software (Siemens Healthcare GmbH, Erlangen, Germany).
skinning function W (.). The pose-dependent shape varia- Note that we use the term organs loosely in this context as
tions BP (θ) : R|θ| 7→ R3N represents the offsets to a tem- it refers to a combined representation of lungs, liver, heart,
plate mesh in the rest pose. Provided a template mesh T̄, a kidneys, bladder, rectum, esophagus, aorta, and the bowel
morphed model can be represented as region. From the segmented volumes we extracted skin Sis ,
bone Sib , and organ Si,k
o surface meshes using the Marching
M = W (T̄ + BS (β) + BP (θ), θ, J (β), W). (1)
Cubes [32] algorithm. Here i ∈ [1, |S|] represents the vol-
ume index and k represents the organ index. Fig. 1 shows
In order to achieve smooth deformations, the Linear
an example surface mesh of a patient. Further, we manu-
Blend Skinning (LBS) W (.) method is used. This works
ally annotated around 60 landmarks Lij both on the skin and
by assigning blend weights W to each vertex of the mesh,
on the bones with the bone’s structure as reference guides.
which determine how much each segment of the skeleton af-
This also acts as a guide in determining the availability of
fects the vertex’s rotation. The resulting transformed vertex
segments such as the arms and legs.
position vi0 is given by the sum of the contributions from
each segment 2.3. Skin Model
K
X In order to create a consolidated human model, it is fun-
vi0 = wk,i Gk vi damental that we establish dense correspondence between
k=1 all scans in the dataset. The typical approach of achiev-
 Sis for any given vertex in M for Ed (Ss , Ms ) and inversely
for Ed (Ms , Ss ). For robustness, we discard matches where
the angle between the corresponding normals are above a
threshold of 30◦ and when the distance between the points
are greater than 30 mm by setting pj in (3) to either 1 or 0.
We make use of the robust Geman-McClure function [34],
represented by ρ to handle noisy data. The data-loss alone
could fit the two surfaces if they are close and and on com-
parable poses, else the optimization process may get stuck
in a local minima. The landmark-loss penalizes misalign-
ment between the set of manually annotated landmarks Lij
of each scan with the corresponding vertices L(M ) of the
skin model. In addition to Elm and Edata we further add
regularization in the form of pose Eθ , shape Eβ , weight Ew
and height prior Eh . The pose prior is the same as Eq. 5
(a) (b) from [35]. Un-natural poses especially for the arms could
Figure 1. Example surface mesh with landmarks (depicted in the
lead to lower data error, hence these terms tries to keep the
image with a blue cross) from the Visceral dataset [30] of (a) skin poses in a realistic range. We make use of the Gaussian
and (b) bone-organ obtained from a segmented CT scan. mixture model provided by [35] for the pose prior. As the
patient are lying on a table in a supine position, whereas
the SMPL model was trained on standing pose we further
ing this is to start from a common template mesh which add an exponential pose loss along the sagital plane for the
ensures identical mesh topology for all the scans [26, 33]. joints near
P the thorax and abdomen region. The shape prior
This is accompanied by a skeleton-based deformation of the Eβ = kβk forces the shapes to be close to the mean
template to estimate the rough pose and shape of the scan. shape. For the partial-body scans, we add additional prior
Subsequently a non-rigid deformation technique is tasked to loss Eh and Ew , when the patients height and weight are
fit the template to the surface scan. The two stage process known. The height and weight of the skin model are mea-
of registering reduces the convergence speed and improves sured in the rest pose for a given shape parameter β. The
accuracy drastically. To avoid falling onto a local minima height is measured from the head to toe, whereas the weight
we make use of previously described manual landmarks to is measured as function of the mesh volume V (M ) as de-
achieve the initial pose deformation of the template. scribed in [36].

2.3.1 Skin Registration Ew = ||(V (Msi ) + 4.937)/1.015 − wi ||22 . (5)

With the availability of a trained SMPL model, the registra- We optimize the following energy term
tion process is simplified due to the preexistence of a SSM.
Hence, the pose and shape can be optimized simultaneously arg min(Elm +Edata +Eθ +Eβ +Ew +Eh ),
by minimizing Elm + Edata , where Elm is the landmark- β,θ,t
(6)
loss (Eq. 4) and Edata the data-loss (Eq. 2).
to obtain an initial fit Ṽsi . The weight terms λx associated
with the energy term Ex is omitted for easier readability.
Edata = λd1 Ed (Sis , Ms ) + λd2 Ed (Ms , Sis ) (2) The initial fit is obtained by first by minimizing the global
translation t, followed by the pose θ, shape β and finally all
three parameters simultaneously.
X If the scans Sis were in a standing pose, registration
pj ρ ||mj − N (S)||22


Ed (S, M (β, θ, t; Φ)) = would be achieved under the assumption that the SMPL
mj ∈(M )
model represents an adequate space of human shape vari-
(3)
ations. However, the model we would like to represent are
X the ones in a supine position, taken during a CT procedure.
Elm = klj − Lj k1 (4)
This usually causes the backs to become flat, stomach to
lj ∈L(M )
be depressed and the chest bulged out. Hence, we perform
The data loss accounts for the distance between the skin non-rigid registration on the initial fit Ṽsi . Similar to the
SMPL model M and the surface Sis . As correspondence is works from [24,37], we represent a set of 3 × 4 affine trans-
not present implicitly, we select the nearest neighbour N in formation matrices Aij associated with each vertex of the
 i
initial fit Ṽs,k , with the aim to align the vertices to the scan
i
Ss . This is achieved by minimizing Elm +Edata +Es +Eo ,
where Es and Eo are smoothing and orthogonality con-
straints [37] respectively. To achieve local rigidity, the
i
affine transformations applied on the vertices Ṽs,k need to
be close to the transformations on the neighbouring vertices
i i
Ṽs,k ∈ N (Ṽs,j ). Therefore, the smoothness term Es (Ṽsi )
can be defined as

2
X
Es (p) = cij kAj pj − Ak pk k2 . (7)
{j,k|{pj ,pk }∈edges(p)}

Here, cij represents the Laplacian cotangent edge weights,

which tries to make changes on the transformation matrices
Aij over the mesh as smooth as possible [38]. The orthog-
onality constraint additionally preserves local rigidity dur-
ing registration by enforcing the affine transformation to be
close to a rigid transformation by
Figure 2. Template bone-organ model, consisting of lungs, liver,
X 2 kidneys, spleen, heart along with aorta and bladder
Eo = Aij − Rij F . (8)
j

Here Rj is the closest projection of Aj onto the rotation of which 65, 617 vertices are made up of the bone section
matrix group. This can be extracted by performing Singu- and the remaining for the internal organs.
lar Value Decomposition on the transformation matrix. All On top of the template, we define a kinematics chain
energies are minimized to obtain a final non-rigid fit Vsi us- made up of Nb = 63 joints comprising 63 segments.
ing a gradient-based LBFGS [39] minimization method and Though we start with 70 individual segments, we consider
make use of automatic differentiation packages. femur-patella and all cervical vertebrae as combined seg-
The main advantage of the two step process of registra- ments. Linear blend skinning is adopted on femur-patella-
tion is that it can handle scans with missing data or holes. tibia and cervical section to achieve a smooth deformable
Missing data here refers to the non-availability of scans sec- bone model. The blend weights are set to 1 for the rigid
tions such as the arms or legs from the partial-body dataset. entity with respect to their own segments, whereas for the
Missing data is identified by the non-availability of land- composite structure it is evenly distributed between the par-
marks for a given scan. As the SMPL model is divided into ent and child segment. The initial blend weights for the
24 sections, we can prevent pose deformation and data loss organs are set only with respect to the vertebral section. We
minimization on those sections. Using a SSM reduces the make use of Blender [41] to automatically generate these
search space, and can provide shape in the realm of proba- weights.
ble human shapes. We rigidly deform the vertebra of the bone-organ mesh
model to one of the segmented CT volumes of comparable
2.4. Bone-Organ Model shape and size, such that the mesh represents a person lay-
Unlike the skin model, publically available SSMs for ing in supine pose. Similarly, we define the skin template
bones and organs do not exist. For this purpose we create a T̃s in supine pose by re-posing the non-rigid skin model to
deformable model from scratch. A template mesh is derived a T-pose of the same CT volume. Additionally, we also ro-
from an existing polygon data BodyParts3D, which was ex- tate the arms and legs of the bone-organ model, such that
tracted from full body MRI images [40]. The bone model is they lay inside and follow the same T-pose as the skin from
made up of 70 segments, including skull, femur, humerus, the SMPL model. The final template bone-organ template
forearm, lower leg, scapula, clavicle, sternum, hands, feet, mesh T̃bo is shown in Fig. 2 .
vertebra, ribs, and pelvis. The organ model includes lungs,
liver, kidneys, spleen, heart, bladder, rectum, esophagus, 2.4.1 Bone Registration
and aorta. We also incorporate the bowel region containing
the stomach and intestines. However, segmentation for the The registration process in general follows the methodology
individual bowel components are not available, rather a hull as described in Sec. 2.3. However, estimating the rough
enclosing the stomach and intestines. The entire bone-organ pose followed by non-rigid registration is not feasible by
template is made up of 104, 546 vertices and 209, 418 faces, virtue of the complex thin structure of bones. This problem
is particularly evident on the scapula and clavicle, which generate an estimated fit of the bones in the case of missing
leads to incorrect poses for the rest of the template. To ad- data around the leg and arm regions. Here, we use a learnt
dress this, we simultaneously estimate a rough shape and mapping between the skin vertices and joints of skeletons
pose. The shape variations are achieved by applying a scale J (Ms ), in particular for the arms, legs, hand and feet. The
transform along a segment in world coordinates. Conse- mapping is learnt on the set of registered scans where afore-
quently, the joint locations along the kinematic chain are mentioned sections where present in the CT data. Using the
also scaled by the same amount. registered skin as a reference, we minimize the loss between
Hereby a simplistic deformable bone model can be ex- the predicted joint locations J (Ms ) and the joint locations
pressed as M b (β̂ b , θ b , tb ; Φb ), where β̂ b ∈ R63×3 repre- obtained from the skeleton model M b (β b , θ b ).
sents the scaling parameters, θ b ∈ R63×3 represents the
pose parameters, tb ∈ R63×3 represents the individual seg- X
ment translation parameters, and Φb represent the model Edata = (λd1 Ed (smk , ssk ) + λd2 Ed (ssk , smk ))
parameters comprising of the kinematic chain and the ini- {smk ,ssk ∈segments(Mb ,Sb )}
tial blend weights. (10)

Similar to the skin, non-rigid registration is performed

for the bones by minimizing Elm + Edata + Es + Eo . Un-
like skin, the bones are narrow structures, resulting in se-
vere mesh overlap when using the same formulation. To
prevent this, we create inter-segment and intra-segment vir-
tual edges on the template mesh as shown in Fig.3. These
virtual edges are defined based on a distance threshold and
direction of the vertex normals. For inter-segment edges the
Figure 3. Example of the virtual edges between two vertebral seg- normals between two set of vertices need to be facing each
ments to prevent mesh overlap during non-rigid registration. Blue
other whereas for the intra-segment edges the normals need
and Orange lines represent inter-segment and intra-segment virtual
edges respectively.
to be facing away from each other.
Organ registration is a straightforward process that is
similar to skin registration. The initial fit is obtained during
Similar to the skin registration from Sec. 2.3, registration
the rigid registration of the bones. The non-rigid registra-
of the bones can be performed with a few minor changes as
tion is accomplished through affine transformations applied
described
directly to individual translations. To avoid mesh overlap,
s we include virtual edges between organs. However, the
arg min(Elm +Edata +Eθb +Eβ̂b +Etb +Elm ). (9)
β̂ b ,θ b ,tb bowel region presents a challenge because there is no dis-
tinct segmentation, only a boundary. To address this, we use
The bone surface scans, represented by Sib , are made up a higher regularizer loss and a penetration loss. The pene-
of 26 separate segments including 24 ribs, the sternum, and tration loss calculates if any vertices of the bowel are inside
the rest of the skeletal structure. The set of vertices cor- another body region and penalizes the distance to the near-
responding to a respective segment on the surface scans est boundary. Examples of registered scans are presented in
and template mesh are represented by ss and sm respec- Figure 4.
tively. The data loss Edata term from Eq. 4 is replaced
p(j) 2.5. Model Formulation
with Eq. 10. The pose Eθb = ||θ jb −θ b || and shape
P
P j p(j)
Eβ̂b = ||β̂ b −β̂ b || prior loss forces the registered model With the availability of registered scans Vs and Vbo in
maintain the original template shape and prevent unnatu- the common mesh topology, we can now formulate the de-
ral poses. Here, p(j) denotes the parent joint of joint j. formable shape model for the skin and internal body. It is
The pose prior is defined only for the vertebral section. As necessary that all scans are normalized to the rest pose as
the shape here represents a scaling function, the shape prior defined by their respective templates T̃s and T̄bo in order
forces the scaling to be similar to that of its parent segment. to remove variance related to body articulation. We first
The incorporation of individual translation t allows free perform the unposing (transforming to rest pose) operation
floating segments such as the sternum, scapula, etc. to Pmove on the skin, followed by the internal body. Unposing the
freely. However, we add a translation prior Etb = ||tjb || skin first, allows us to use the skin as a guide for unposing
that prevents large translation movement far away from its the internal body later. From the pose-normalized models,
initial position. In addition to the landmark error Elm , we it is then possible to learn a joint shape-space of the skin-
s
include a skin-based vicinity landmarks Elm . This helps to bone-organ model.
 the joint regressor as |Js |/|||Js |||1 . During the optimiza-
tion, we further add a regularizing term defined as
Sn
X i
||Js (W (Uis , θ̃ s , BPs , Js )) − J (Ṽsi )||2 ,
i

on the joint locations, such that it is close to the joint loca-

tions from the original rigid fit.
Symmetry regularizer is applied on the pose-normalized
mesh vertices Uis and on its joint locations Js (Uis ) to en-
force symmetry along the sagittal plane defined as
Sn
X
||Uis −m(Uis )|| + ||Js (Uis )−m(Js (Uis ))||2 , (12)
i

where m denotes the mirror vertices or mirror joints.

2.5.2 Bone-Organ Unposing

Figure 4. Examples of registered patients. Similar to the skin model, we perform unposing of the bone-
i
organ registered mesh Vbo to the rest pose Uibo . However,
performing exactly the same would lead to incorrect skeletal
2.5.1 Skin Unposing
localization and mesh overlap of the internal body and the
We unpose the registered skin mesh Vsi to the rest pose Uis , skin.
defined by the template skin mesh T̃s . As we prefer the The objective function defined in Eq. 13 is to unpose the
patient skin model to operate similarly to that of the original body-organ model by minimizing the distance between the
i i
SMPL model, we optimize for a new joint regressor Js , model vertices V̄bo =W (Uibo , θ̃ bo ; Jbo , Wbo ) and the regis-
blend weights Ws and pose related shape variations BPs . tration vertices. Additionally, the objective function defined
This is done by minimizing the distance between the skin in Eq. 13 maintains the distance P between the skin and
model vertices and the registration vertices as described in bone-organ model among both posed and unposed states as
the following equation i
shown in Eq. 14. We do this with pose-only θ̄ bo deforma-
i
tion of the unposed model Ūibo =W (Uibo , θ̄ bo ; Jbo , Wbo ).
Sn 2 For each skin vertex of the registered skin mesh Vsi , we find
X i i
arg min W (Uis +BPs (θ̃ s ), θ̃ s ; Js , Ws )−Vsi . at most 6 bone vertices from the registered bone-organ mesh
i i
Uis ,θ̃ s ,BPs ,Js , i Vbo based on empirically chosen distance (based on human
(11) size and the particular bone segment) and vertex normal
During optimization, we initialize Js , Ws and BPs with (skin and bone normal within 30 deg) thresholds. However,
the original SMPL parameters. The rest pose vertices Uis we ignore skin vertices around the complex shoulders, el-
are initialized M (β i ) with only the shape parameters β i bows and knees joints, along with bone vertices for scapula
obtained from the initial rigid skin fit. Similarly, the poses and clavicle. This avoids any pose related influences during
i
θ̃ s are initialized with the poses θ i obtained from the initial optimization around these regions.
rigid skin fit. To stabilize the optimization, we make use of
Sn
multiple regularizers based on various assumptions. X
i i 2
arg min V̄bo −Vbo ; (13)
As, the pose normalized subjects Uis needs to be aligned i
Uibo ,θ̃ bo ,Jbo ,Wbo i
to
PSthe template mesh T̃s , we add an edge loss of the form
i i
i ||Us,e −T̃s,e ||, where Us,e ,T̃s,e ∈ edges(T̃s ) repre-
n

Sn
sents the normalized direction vector for a pair of neigh- X
i arg min ||P(Ūibo , Uis ) − P(V̄bo
i
, Vsi )|| (14)
bouring vertices. We further add constraints on θ̃ s , Js , Ws i i
θ̃ bo ,θ̄ bo ,Jbo ,Wbo i
and BPs in the form of L2 loss, to not deviate too much
from its initial values. In the original SMPL model, the joint We start off by initialising the joint regressor Jbo on the
regressor was computed using non-negative least squares, template bone-organ model Tbo based on the joint locations
with a constraint that the weights add up to 1. We maintain from its initial kinematic chain. The joints are always lo-
similar setting during the joint optimization by normalizing cated between 2 bone segments. We randomly sample 50
 PC1 PC2 PC3
−2σ 2σ −2σ 2σ −2σ 2σ

Figure 5. The first three principal components of body shape are shown, varying about 2 standard deviations after normalizing the variance.
One could infer that the height and weight of the patient are mostly explained by the first two components

closest vertices to the joint from both segments where the weightage to reduce mesh interpenetration. Symmetry reg-
vertex normals approximately faces the vertex-joint direc- ularizer is applied only to the bone structures. Regularizing
tion. The joint regressor Jbo is learnt using a least square the joint locations of bone and skin along the arms and legs
fit for the sampled vertices. The rest pose vertices Uibo are are done by minimizing the following equation:
i i
initialized M bo (β̂ b ) with only the scale parameters β̂ b ob-
tained from the initial rigid bone fit. Similar to the skin ||Jbo (Uibo ) − Js (Uis )||2 .
i
model, the blend weights Wbo and poses θ̃ bo are initialized. We alternate between optimizing Eq. 13 and Eq. 14,
During unposing of the bone-organ model, we define that while carrying the optimized parameters between them. For
i
the motion of ribs, sternum and pelvis are a function of optimization of Eq. 14, we initialize the pose θ̄ bo with zero,
shape rather than a function of the pose. Hence, we ini- and regularize them towards zero. While alternating to op-
tialize the pose to zero for these particular segments. Note timize Eq. 13, we initialize unposed vertices Uibo with the
that, only the ribs and sternum are leaf nodes in our kine- obtained vertices Ūibo after optimizing Eq. 14.
matic chain, i.e. there are no child segments. However, for
the femur, we additionally include the pelvis rotation, as it 2.5.3 Shape Space
is its parent node.
From the unposed skin Uis and bone-organ Uibo volume, we
To stabilize the optimization we make use of similar reg-
learn the shape components with the aid of mean and prin-
ularizes defined in the skin model for both the objectives
cipal shape components. We do not have complete registra-
functions. For the edge loss, we additionally incorporate
tions around skulls, arms and legs for some of the volume.
the virtual edges from the registration process with lower
Hence, we use a publicly available implementation 1 of
Probabilistic Principal Component Analysis (PPCA) [42],
which can handle missing data. By performing PPCA, we

obtain a mean skin Tsµ , bone-organ Tboµ and vertex offsets
 to the mean in the form of shape space for skin Bsµ and Bbo
µ
bone-organ.

9DULDQFH

3. Evaluation

In Fig. 5, we visualize the first three shape components,
 while Fig. 6 displays the cumulative variance of the full
&XPPXODWLYH
model. The first 10 shape components captures 88% of the
 9DULDQFH
variance, and the first 20 components capture 92% of the
        
1RRI6KDSH&RHIILFLHQWV variance. Although the shape space of the skin and bone-
organ are coupled, the kinematic model is separated to ac-
Figure 6. Cumulative variance of the skin-bone-organ model shape count for differences in skeletal posture. We use a neutral
space. 1 https://github.com/allentran/pca-magic
  
 
| |10  PHWDGDWD
 | |50 VNLQVXUIDFH


HUURUPP
HUURUPP

 


 





HU\

HU\
EUD

V
WDO
DUW

HU

ULJ QH\
J
Q
HU

EUD

V
WDO
DUW

HU

ULJ QH\
J
Q
HU
QJ

H\

QJ

H\
OYL

OYL
OXQ
OHH

OXQ
OHH
OLY

GG

OLY

GG
HOH

HOH
LGQ

LGQ
WOX

WOX
KH

KH
DUW

DUW
UWH
SH

UWH
SH
NLG

NLG
KW
VS

KW
VS
EOD

EOD
VN

VN
WN

WN
OHI

OHI
YH

YH
KW

KW
OHI

OHI
ULJ

ULJ
(a) (b)

Figure 7. (a) Box-plot of model generalization on the registered ACRIN dataset using 10 and 50 shape components. (b) Box-plot of model
generalization on the registered ACRIN dataset by only using the metadata and skin surface as a guide to determine the internal body shape.

3.1. Model Generalization

YHUWHEUD OLYHU
 VNHOHWDO VSOHHQ
RUJDQV EODGGHU We register the skin, bones and organs on separate held
 KHDUW DUWHU\ out test set. For the test set we use CT volumes from
NLGQH\V PHDQ
 OXQJV FHQWURLG ACRIN-NSCLC-FDG-PET [43]. We ignore all volumes
where the z-spacing is greater then 3.27 mm, while retain-
HUURUPP

 ing volumes greater than 2.5 mm only if clear separation

 of the vertebra is visible. In total we test on 78 scans,
which were registered to its segmentations as described in

the methodology section.
 We minimize the vertex-to-vertex error between the re-
constructed model and the registration while regularizing

the process with shape and pose priors. The shape prior
      keeps the reconstructed model close to its mean, while the
1XPEHURIFRPSRQHQWV
pose prior restricts individual vertebrae from deviating too

Figure 8. Residual error on ACRIN dataset with respect to the dif-

ferent number of shape components. The centroid represents, the
mean error of the centre of mass for individual segments between 
 | |10
reconstructed and registered models. | |50


model rather than separating into two genders on basis of 

HUURUPP

the number of CT volumes available. We map the gender,


height, and weight of the registered patients where available
to the shape space coefficients using a linear regressor. We 
provide qualitative results of random samples generated in

Fig. 11.
To demonstrate the effectiveness of the skin-bone-organ 
model we perform evaluations on multiple datasets. First

we assess the model generalizability on a registered test set.

















7
/
/
/
/
/
7

7
7
7
7
7
7
7
7
7
7

We then demonstrate model completion from metadata such 9HUWHEUD

as height, weight, or surface scans. We finally evaluate the
model performance on various public datasets, where seg- Figure 9. Model generalization on VerSe dataset using 10 and 50
mentations of organs or bones are provided. shape components.
 
  
| |10  | |10 | |10
 | |50 | |50 
| |50



HUURUPP
HUURUPP



HUURUPP


 


 




H\
HU

Q
H\
O

V
V

FUD

OHH
OYL
L

OLY
DUW

J
QJ

GQ
GQ
HOY

OXQ
SH

VS
VD

WOX

NL
KH

L
WS

WN

KW
KW

KW

OHI
OHI

OHI

ULJ
ULJ

ULJ
(a) CTPEL (b) SturctSeg (c) Abdomen1k

Figure 10. Model generalization on various datasets using 10 and 50 shape components.

much from their parent and child nodes. It’s important to 3.2. Model Generalization on Public Organ Seg-
note that we do not translate individual segments, and all the mentations
shape variations are accounted for only by the shape space
We evaluate the generalization of our model on public
of the model. In the case of the ACRIN dataset, regions
datasets containing multiple organs or bones. Specifically,
such as the legs, arms, and part of the skull are not avail-
we test our model on Verse [44] for vertebra, CTPEL [45]
able, so we exclude these regions while measuring metrics
for pelvis and sacral, Abdomen1K [46] for liver, kidney, and
and during reconstruction.
spleen, and StructSeg [47] for lungs and heart. For all meth-
ods, we follow a similar registration process. To register to
In Figure 7a, we present the box plot illustrating the er- each dataset, we obtain a surface mesh of the segmented
ror in individual segments for 10 and 50 shape components. volume using marching cubes [32], and then minimize the
The vertebra includes all vertebral sections, the pelvis in- chamfer loss between the vertices of our model and the tar-
cludes the pelvis and sacral, and the skeleton comprises the get surface mesh. For regularization we use the same pro-
remaining bone structures. In Figure 8, we display the gen- cedure as defined in Sec. 3.1 by using pose and shape prior
eralization error across different numbers of shape compo- with no individual translations, i.e. the combined shape
nents. The results indicate that the bones can be generalized model has to best represent all target surfaces. The final
well, with an error of 3.66 mm at 25 components. However, errors presented are the bidirectional point-to-surface dis-
the errors for individual organs are significantly higher, as tance [17].
we optimize the model for the entire body rather than indi- The model’s generalization performance for the verte-
vidual organs. This is expected since the shape space ac- brae region on the Verse dataset is presented in Fig.9. The
counts for variations in organ shapes and locations. Using full Verse dataset, including the train, test and validation
only 25 components, we achieve an average organ error of sets, was used for testing. However, volumes containing L6
8.83 mm, which can provide a starting point for downstream and T13 vertebrae, which are not represented by our model,
tasks involving organ shape and placement. We also mea- were excluded. The generalization errors for the pelvis and
sure the mean centroid error for each segment, which rep- sacral region in CTPEL, for the liver, kidney, and spleen in
resents the center of mass error. With 25 components, we StructSeg, and for the lungs and heart of the Abdomen1k
observe a mean centroid error of 2.52 mm. dataset are presented in Fig. 10. In the Abdomen1k dataset,
volumes containing incomplete liver were excluded from
the analysis. The results obtained on these datasets are con-
Fig. 7b displays the ability of our model to estimate the sistent with those observed on the ACRIN dataset.
shape of inner organs and skeleton using patient metadata
or skin surface. For estimating with metadata, we use the 4. Conclusion
linear regressor with patient’s gender, weight, and height to
determine the model shape coefficients. On the other hand, A digital twin of a patient can be a valuable tool to
when using the skin surface, we only use the shape coef- improve multiple clinical tasks, such as automating clini-
ficients of our skin model that fit the target skin. In both cal workflows, estimating patient-specific X-ray dose ex-
cases, we optimize only for the pose while using the previ- posure to patient and medical staff, positioning, markerless
ously determined shape coefficients. We observe an average tracking and navigation assistance in image-guided inter-
overall error of 8.11 mm and 8.68 mm using metadata and ventions. For the first time, we provide a joint model with
skin surface, respectively. an accurate estimation of shapes and poses of the patient
 Figure 11. Random samples drawn from the model.

surface, skeleton and internal organs. We propose a de- [6] A. Maier, H. G. Hofmann, C. Schwemmer, J. Horneg-
formable shape and pose human model devised out of indi- ger, A. Keil, and R. Fahrig, “Fast simulation of x-
vidual segments such as the skin, internal organs and bones, ray projections of spline-based surfaces using an ap-
learnt from surfaces extracted from segmented whole-body pend buffer,” Physics in Medicine & Biology, vol. 57,
CT images. With surface errors of 3.66 mm for bones and no. 19, p. 6193, 2012.
an average organ error of 8.83 mm, we believe that the sta-
tistical model will allow many automation tasks in clinical [7] T. Heimann and H.-P. Meinzer, “Statistical shape
workflows and lays the basis for fast intraoperative model models for 3d medical image segmentation: A re-
personalisation given patient-specific information. view,” Medical Image Analysis, vol. 13, no. 4, pp. 543
– 563, 2009.
Disclaimer
[8] F. Ambellan, A. Tack, M. Ehlke, and S. Zachow, “Au-
The concepts and information presented in this article tomated segmentation of knee bone and cartilage com-
are based on research and are not commercially available. bining statistical shape knowledge and convolutional
neural networks: Data from the osteoarthritis initia-
References tive,” Medical Image Analysis, vol. 52, no. 2, pp. 109
[1] M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, – 118, 2019.
and M. J. Black, “SMPL: A skinned multi-person lin-
ear model,” ACM Trans. Graphics (Proc. SIGGRAPH [9] M. Esfandiarkhani and A. H. Foruzan, “A general-
Asia), vol. 34, no. 6, pp. 248:1–248:16, Oct. 2015. ized active shape model for segmentation of liver in
low-contrast ct volumes,” Computers in Biology and
[2] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, Medicine, vol. 82, pp. 59–70, 2017.
J. Rodgers, and J. Davis, “Scape: shape completion
and animation of people,” in ACM SIGGRAPH 2005 [10] H. Seim, D. Kainmueller, H. Lamecker, M. Binder-
Papers, 2005, pp. 408–416. nagel, J. Malinowski, and S. Zachow, “Model-based
auto-segmentation of knee bones and cartilage in mri
[3] P. Roser, X. Zhong, A. Birkhold, N. Strobel, data,” in Proc. MICCAI Workshop Medical Image
M. Kowarschik, R. Fahrig, and A. Maier, “Physics- Analysis for the Clinic, 2010, pp. 215–223.
driven learning of x-ray skin dose distribution in in-
terventional procedures,” Medical Physics, vol. 46, [11] X. Zhong, M. Amrehn, N. Ravikumar, S. Chen,
no. 10, pp. 4654–4665, 2019. N. Strobel, A. Birkhold, M. Kowarschik, R. Fahrig,
[4] P. B. Johnson, D. Borrego, S. Balter, K. Johnson, and A. Maier, “Deep action learning enables robust
D. Siragusa, and W. E. Bolch, “Skin dose mapping 3d segmentation of body organs in various ct and mri
for fluoroscopically guided interventions,” Medical images,” Scientific Reports, vol. 11, no. 1, pp. 1–14,
Physics, vol. 38, no. 10, pp. 5490–5499, 2011. 2021.

[5] M. Ehlke, H. Ramm, H. Lamecker, H.-C. Hege, and [12] N. Baka, B. L. Kaptein, M. de Bruijne, T. van Walsum,
S. Zachow, “Fast generation of virtual x-ray images J. Giphart, W. J. Niessen, and B. P. Lelieveldt, “2d–3d
for reconstruction of 3d anatomy,” IEEE transac- shape reconstruction of the distal femur from stereo x-
tions on visualization and computer graphics, vol. 19, ray imaging using statistical shape models,” Medical
no. 12, pp. 2673–2682, 2013. image analysis, vol. 15, no. 6, pp. 840–850, 2011.
[13] A. Gilbert, M. Marciniak, C. Rodero, P. Lamata, imaging research and optimization,” Medical Physics,
E. Samset, and K. Mcleod, “Generating synthetic la- vol. 42, no. 8, pp. 4719–4726, 2015.
beled data from existing anatomical models: an ex-
ample with echocardiography segmentation,” IEEE [23] W. P. Segars, J. Bond, J. Frush, S. Hon, C. Eckers-
Transactions on Medical Imaging, vol. 40, no. 10, pp. ley, C. H. Williams, J. Feng, D. J. Tward, J. T. Rat-
2783–2794, 2021. nanather, M. I. Miller, D. Frush, and E. Samei, “Pop-
ulation of anatomically variable 4d xcat adult phan-
[14] S. Amirrajab, S. Abbasi-Sureshjani, Y. Al Khalil, toms for imaging research and optimization,” Medical
C. Lorenz, J. Weese, J. Pluim, and M. Breeuwer, Physics, vol. 40, no. 4, p. 043701, 2013.
“Xcat-gan for synthesizing 3d consistent labeled car-
diac mr images on anatomically variable xcat phan- [24] B. Allen, B. Curless, and Z. Popović, “Articulated
toms,” in International Conference on Medical Im- body deformation from range scan data,” ACM Trans-
age Computing and Computer-Assisted Intervention. actions on Graphics (TOG), vol. 21, no. 3, pp. 612–
Springer, 2020, pp. 128–137. 619, 2002.
[15] S. L. Delp, F. C. Anderson, A. S. Arnold, P. Loan, [25] D. Hirshberg, M. Loper, E. Rachlin, and M. Black,
A. Habib, C. T. John, E. Guendelman, and D. G. The- “Coregistration: Simultaneous alignment and model-
len, “Opensim: open-source software to create and an- ing of articulated 3D shape,” in European Conf. on
alyze dynamic simulations of movement,” IEEE trans- Computer Vision (ECCV), ser. LNCS 7577, Part IV.
actions on biomedical engineering, vol. 54, no. 11, pp. Springer-Verlag, Oct. 2012, pp. 242–255.
1940–1950, 2007.
[26] N. Hasler, C. Stoll, M. Sunkel, B. Rosenhahn, and H.-
[16] R. Kelc, “Zygote body: a new interactive 3-
P. Seidel, “A statistical model of human pose and body
dimensional didactical tool for teaching anatomy,”
shape,” in Computer graphics forum, vol. 28, no. 2.
2012.
Wiley Online Library, 2009, pp. 337–346.
[17] D. Meng, M. Keller, E. Boyer, M. Black, and S. Pu-
jades, “Learning a statistical full spine model from [27] L. Pishchulin, S. Wuhrer, T. Helten, C. Theobalt, and
partial observations,” in Shape in Medical Imag- B. Schiele, “Building statistical shape spaces for 3d
ing. Springer International Publishing, Oct. 2020, human modeling,” Pattern Recognition, vol. 67, pp.
pp. 122,133. 276–286, 2017.

[18] P. Kadlecek, A.-E. Ichim, T. Liu, J. Krivanek, and [28] K. M. Robinette, S. Blackwell, H. Daanen,
L. Kavan, “Reconstructing personalized anatomical M. Boehmer, and S. Fleming, “Civilian ameri-
models for physics-based body animation,” ACM can and european surface anthropometry resource
Trans. Graph., vol. 35, no. 6, 2016. (caesar), final report. volume 1. summary,” Sytronics
Inc Dayton Oh, Tech. Rep., 2002.
[19] M. Keller, S. Zuffi, M. J. Black, and S. Pujades,
“OSSO: Obtaining skeletal shape from outside,” in [29] R. M. Murray, Z. Li, S. S. Sastry, and S. S. Sastry,
Proceedings IEEE/CVF Conf. on Computer Vision A mathematical introduction to robotic manipulation.
and Pattern Recognition (CVPR), Jun. 2022, pp. CRC press, 1994.
20 492–20 501.
[30] M. Krenn, K. Grünberg, O. Jimenez-del Toro,
[20] W. P. Segars, G. Sturgeon, S. Mendonca, J. Grimes, A. Jakab, T. Salas Fernandez, M. Winterstein, M.-A.
and B. M. W. Tsui, “4d xcat phantom for multimodal- Weber, and G. Langs, Datasets Created in VISCERAL.
ity imaging research,” Medical Physics, vol. 37, no. 9, Cham: Springer International Publishing, 2017, pp.
pp. 4902–4915, 2010. 69–84.
[21] M. J. Ackerman, “The visible human project: a re-
source for education.” Academic medicine: journal [31] R. R. Beichel, E. J. Ulrich, C. Bauer, A. Wahle,
of the Association of American Medical Colleges, B. Brown, T. Chang, K. Plichta, B. Smith,
vol. 74, no. 6, pp. 667–670, 1999. J. Sunderland, T. Braun, A. Fedorov, D. Clu-
nie, M. Onken, V. A. Magnotta, Y. Menda,
[22] W. P. Segars, H. Norris, G. M. Sturgeon, Y. Zhang, J. Riesmeier, S. Pieper, R. Kikinis, M. Gra-
J. Bond, A. Minhas, D. J. Tward, J. T. Ratnanather, ham, T. Casavant, M. Sonka, and J. Buatti,
M. I. Miller, D. Frush, and E. Samei, “The develop- “Data from qin-headneck,” 2015. [Online]. Available:
ment of a population of 4d pediatric xcat phantoms for https://wiki.cancerimagingarchive.net/x/qQ9p
[32] W. E. Lorensen and H. E. Cline, “Marching cubes: D. Štern, M. Urschler, M. Chen, D. Cheng, N. Less-
A high resolution 3d surface construction algorithm,” mann, Y. Hu, T. Wang, D. Yang, D. Xu, F. Ambellan,
ACM siggraph computer graphics, vol. 21, no. 4, pp. T. Amiranashvili, M. Ehlke, H. Lamecker, S. Lehnert,
163–169, 1987. M. Lirio, N. P. de Olaguer, H. Ramm, M. Sahu,
A. Tack, S. Zachow, T. Jiang, X. Ma, C. Angerman,
[33] B. Allen, B. Curless, and Z. Popović, “The space of X. Wang, K. Brown, A. Kirszenberg, É. Puybareau,
human body shapes: reconstruction and parameteriza- D. Chen, Y. Bai, B. H. Rapazzo, T. Yeah, A. Zhang,
tion from range scans,” ACM transactions on graphics S. Xu, F. Hou, Z. He, C. Zeng, Z. Xiangshang,
(TOG), vol. 22, no. 3, pp. 587–594, 2003. X. Liming, T. J. Netherton, R. P. Mumme, L. E.
Court, Z. Huang, C. He, L.-W. Wang, S. H. Ling,
[34] S. Geman and D. E. McClure, “Statistical methods for
L. D. Huynh, N. Boutry, R. Jakubicek, J. Chmelik,
tomographic image reconstruction,” 1987.
S. Mulay, M. Sivaprakasam, J. C. Paetzold, S. Shit,
[35] F. Bogo, A. Kanazawa, C. Lassner, P. V. Gehler, I. Ezhov, B. Wiestler, B. Glocker, A. Valentinitsch,
J. Romero, and M. J. Black, “Keep it smpl: Automatic M. Rempfler, B. H. Menze, and J. S. Kirschke, “VerSe:
estimation of 3d human pose and shape from a single A vertebrae labelling and segmentation benchmark for
image,” in ECCV, 2016. multi-detector CT images,” Medical Image Analysis,
vol. 73, p. 102166, Oct. 2021. [Online]. Available:
[36] S. Ice, “Obesity in perspective (volume 2, parts 1 and https://doi.org/10.1016/j.media.2021.102166
2 of fogarty international center series on preventive
medicine),” Psychosomatic Medicine, vol. 39, no. 3, [45] B. Connolly and C. Wang, “Segmented ct pelvis
1977. scans with annotated anatomical landmarks,” 2019.
[Online]. Available: https://datahub.aida.scilifelab.se/
[37] K. Li, J. Yang, Y.-K. Lai, and D. Guo, “Robust non- 10.23698/aida/ctpel
rigid registration with reweighted position and trans-
formation sparsity,” IEEE Transactions on Visualiza- [46] J. Ma, Y. Zhang, S. Gu, C. Zhu, C. Ge, Y. Zhang,
tion and Computer Graphics, vol. 25, pp. 2255–2269, X. An, C. Wang, Q. Wang, X. Liu, S. Cao, Q. Zhang,
2019. S. Liu, Y. Wang, Y. Li, J. He, and X. Yang,
“Abdomenct-1k: Is abdominal organ segmentation a
[38] O. Sorkine, “Laplacian mesh processing.” solved problem?” IEEE Transactions on Pattern Anal-
ysis and Machine Intelligence, 2021.
[39] D. C. Liu and J. Nocedal, “On the limited memory
BFGS method for large scale optimization,” vol. 45, [47] Hongsheng Li and M. Chen, “Automatic structure
no. 1-3, pp. 503–528, Aug. 1989. [Online]. Available: segmentation for radiotherapy planning challenge
https://doi.org/10.1007/bf01589116 2020,” 2020. [Online]. Available: https://zenodo.org/
record/3718884
[40] N. Mitsuhashi, K. Fujieda, T. Tamura, S. Kawamoto,
T. Takagi, and K. Okubo, “Bodyparts3d: 3d structure
database for anatomical concepts,” Nucleic acids re-
search, vol. 37, no. suppl 1, pp. D782–D785, 2009.

[41] B. O. Community, Blender - a 3D modelling and

rendering package, Blender Foundation, Stichting
Blender Foundation, Amsterdam, 2018. [Online].
Available: http://www.blender.org

[42] M. E. Tipping and C. M. Bishop, “Probabilistic prin-

cipal component analysis,” Journal of the Royal Sta-
tistical Society: Series B (Statistical Methodology),
vol. 61, no. 3, pp. 611–622, 1999.

[43] P. Kinahan, M. Muzi, B. Bialecki, B. Herman, and

L. Coombs, “Data from the acrin 6668 trial nsclc-fdg-
pet,” 2019.

[44] A. Sekuboyina, M. E. Husseini, A. Bayat, M. Löffler,

H. Liebl, H. Li, G. Tetteh, J. Kukačka, C. Payer,