phantomAI
phantomAI
3, MARCH 2024
Abstract — Accurate measurement of optical absorption depth-dependent loss of signal intensity. We demonstrate
coefficients from photoacoustic imaging (PAI) data would that training on experimental phantoms can restore the
enable direct mapping of molecular concentrations, pro- correlation of signal amplitudes measured in depth. While
viding vital clinical insight. The ill-posed nature of the the absolute quantification error remains high and further
problem of absorption coefficient recovery has prohibited improvements are needed, our results highlight the promise
PAI from achieving this goal in living systems due to the of deep learning to advance quantitative PAI.
domain gap between simulation and experiment. To bridge
this gap, we introduce a collection of experimentally well- Index Terms— Deep learning, image reconstruction, pho-
characterised imaging phantoms and their digital twins. toacoustics, quantitative imaging.
This first-of-a-kind phantom data set enables supervised
I. I NTRODUCTION
training of a U-Net on experimental data for pixel-wise
estimation of absorption coefficients. We show that train-
ing on simulated data results in artefacts and biases in
the estimates, reinforcing the existence of a domain gap
P HOTOACOUSTIC (PA) imaging (PAI) has the unique
ability to provide high-resolution optical-contrast molec-
ular imaging up to several centimetres deep in tissue with
between simulation and experiment. Training on experimen- potential clinical application areas ranging from diagnosis of
tally acquired data, however, yielded more accurate and
robust estimates of optical absorption coefficients. We com- breast cancer [1] or cardiovascular disease [2], to grading of
pare the results to fluence correction with a Monte Carlo skin [3], [4] or inflammatory diseases [5]. PAI transforms
model from reference optical properties of the materials, absorbed laser energy into acoustic sound waves [6], [7]
which yields a quantification error of approximately 20%. that can be measured as time-series pressure data p(t) using
Application of the trained U-Nets to a blood flow phantom acoustic detectors [8], [9].
demonstrated spectral biases when training on simulated
data, while application to a mouse model highlighted the A key aim of PAI is estimation of absolute molecular con-
ability of both learning-based approaches to recover the centrations from images acquired at multiple wavelengths [10],
[11]. To achieve this goal, two inverse problems have to
Manuscript received 18 September 2023; accepted 5 November 2023. be solved: the acoustic inverse problem, which refers to the
Date of publication 8 November 2023; date of current version 4 March
2024. This work was supported by the NVIDIA Academic Hardware Grant reconstruction of the initial pressure distribution p0 from
Program and utilized two Quadro Ray Tracing Texel eXtreme (RTX) 8000. p(t) and the optical inverse problem, which refers to the
The work of Janek Gröhl was supported by the Deutsche Forschungs- estimation of the optical absorption coefficient µa from p0 .
gemeinschaft (DFG, German Research Foundation) under project GR
5824/1. The work of Lina Hacker was supported by the MedAccel Pro- Reconstructing p0 from p(t) requires an inverse acoustic
gramme of National Physical Laboratory (NPL) financed by the Industrial operator, denoted as A→1 (an image reconstruction algorithm).
Strategy Challenge Fund of the Department for Business, Energy and To obtain accurate reconstructions, the acoustic properties of
Industrial Strategy. The work of Thomas R. Else, Paul W. Sweeney,
and Sarah E. Bohndiek was supported by the Cancer Research U.K. the tissue, ω, and the characteristics of the acoustic detector,
under Grant C9545/A29580. The work of Ellie V. Bunce was supported ε, must be taken into consideration: p0 ↑ A→1 ( p(t, ε), ω, ε).
by the Cancer Research U.K. (Radiation Research Network (Rad- Once reconstructed, determining µa from p0 , for a given
Net) Cambridge) under Grant C17918/A28870. (Corresponding author:
Janek Gröhl.) wavelength ω at position x, must deal with the fact that
This work involved human subjects or animals in its research. Approval p0 depends not only on the local absorption coefficient µa ,
of all ethical and experimental procedures and protocols was granted by but also the temperature (T )-dependent Grüneisen parameter
the Project and Personal Licences under PPL No. PE12C2B96, issued
under the United Kingdom Animals (Scientific Procedures) Act, 1986 and ϑ and the light fluence ϖ, which is a function of both µa and
approved locally under compliance form CFSB2317. the optical scattering coefficient µs : p0 (x, ω, T ) = ϑ(x, T ) ·
Janek Gröhl, Thomas R. Else, Ellie V. Bunce, Paul W. Sweeney, and µa (x, ω) · ϖ(x, ω, µa (ω), µs (ω)).
Sarah E. Bohndiek are with the Cancer Research U.K. Cambridge
Institute, CB2 0RE Cambridge, U.K., and also with the Department The spatial distribution of ϖ and ϑ in living subjects are
of Physics, University of Cambridge, CB3 0HE Cambridge, U.K. typically unknown but must be accounted for to accurately
(e-mail: [email protected]; [email protected]; [email protected]; determine µa . ϑ is often assumed constant [11]. ϖ could be
[email protected]; [email protected]).
Lina Hacker was with the Department of Physics and the accounted for by iterative methods, using fixed-point iteration
Cancer Research U.K. Cambridge Institute, University of Cambridge, schemes assuming a priori known µs [12] or variational
CB3 0HE Cambridge, U.K. She is now with the Department of approaches, allowing a range of constraints and priors [13].
Oncology, University of Oxford, OX3 7DQ Oxford, U.K. (e-mail:
[email protected]). To date, these methods have been restricted to simulated data,
Digital Object Identifier 10.1109/TMI.2023.3331198 except when including a calibrated imaging target into the
© 2023 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License.
For more information, see https://creativecommons.org/licenses/by/4.0/
GRÖHL et al.: MOVING BEYOND SIMULATION: DATA-DRIVEN QUANTITATIVE PHOTOACOUSTIC IMAGING 1215
medium [14]. Deep learning approaches have been proposed and µa = 0.5 cm→1 – 4.0 cm→1 in the inclusions. Reduced
to solve both the acoustic [15], [16] and optical inverse prob- scattering was set to be µ↔s = 5 cm→1 – 15 cm→1 , defined at
lems [17], [18], [19] in PAI. Iterative or variational methods a reference wavelength of 800 nm. We attempted to achieve
typically work well with simulated data, but do not achieve a uniform distribution over this range for the inclusions, but
convincing results on experimental data [20]. A key limitation due to experimental constraints and the exclusion of phantoms
for application in living subjects is that the optical and acoustic from a manual quality assurance step, this was not always
properties are unknown, making validation of quantitative PAI possible. For the background we initially created a uniform
methods extremely difficult. One indirect possibility to validate distribution of absorption coefficients and complemented this
these methods is to analyse the performance of methods that later on with more phantoms with a background absorption
use the estimated µa as an input, for example, semantic image of 0.1 cm→1 when extending the dataset, leading to a skewed
segmentation [21], [22], or oximetry [23]. Li et al. [24] have distribution. The phantoms featured various inclusion patterns
recently proposed an approach to enable training on simulated with distinct µa and µ↔s combinations in both the inclusions
data with known ground truth, using a cycle-GAN network to and the backgrounds.
transform the simulation data distribution into the experimental
domain, enabling them to train a network on transformed B. Optical Property Characterisation
simulated data and apply it to previously unseen experimental For each phantom material, rectangular samples with a
data. length of 5.9 cm, a width of 1.8 cm, and a thickness ranging
Here, we bridge the gap from simulated to experimental between 2 mm and 4 mm were prepared for optical character-
training data by fabricating a collection of well-characterised isation. Sample thicknesses were determined at five distinct
co-polymer-in-oil tissue-mimicking phantoms [25] covering locations using digital vernier calipers. We used an in-house
a biologically relevant range of µa and µs , to enable the use double integrating sphere (DIS) system [25] based on the
of supervised deep learning for the optical inverse problem system developed by Pickering et al. [28] to determine µa and
of PAI. We create digital phantom twins by characterising µ↔s of the phantom material in a wavelength range of 600 to
optical properties with a double integrating sphere (DIS) 950 nm. The recorded transmittance and reflectance values
system and train U-Nets [26] to estimate µa on matching were normalised and analysed with the inverse adding dou-
simulated and experimental PA images. We apply the trained bling (IAD) algorithm [29] to estimate the optical properties
U-Nets to a held-out set of test phantoms and compare the of the material. The anisotropy (g) and refractive index (n)
results to fluence correction with a Monte Carlo model based were assumed to be g = 0.7 and n = 1.4 [30].
on the DIS reference measurements. Importantly, we then
explore the potential of the methods on pre-clinical data from C. Photoacoustic Imaging and Data Processing
living subjects. The phantoms were imaged using a pre-clinical PAI sys-
tem (MSOT inVision-256TF, iThera Medical GmbH, Munich,
II. M ETHODS Germany). It has 256 transducer elements in a circular array
A. Tissue Mimicking Phantoms with an angular coverage of 270↓ which have a 5-MHz
centre frequency and 60% bandwidth. More details on the
A total of 137 cylindrical phantoms with a diameter of
measurement setup can be found in a prior publication [31].
27.5 mm, a height of 65-80 mm, and an approximate volume
We imaged in the wavelength range from 700 nm to 900 nm
of 40-50 mL were fabricated based on a previously published
in 10 nm steps using 10 frame averaging. The phantoms were
protocol [25]. Briefly, 83.80 g mineral oil (Sigma Aldrich-
mounted in the imaging system using a custom 3D-printed
330779-1L) was used as a base. 25.14 g polystyrene-block-
polylactic acid phantom holder printed with the Prusa i3
poly(ethylene-ran-butylene)-block-polystyrene (SEBS, Sigma
MK3S+ (Prusa Research, Czechia) and immersed in deionized
Aldrich 200557-250G) as the polymer and 2.09 g butylated
water for acoustic coupling. The phantoms were imaged at
hydroxytoluene (HT, Sigma Aldrich W218405-1KG-K) as an
different cross-sections in 2 mm steps along a 44 mm-long
antioxidant was added to the mineral oil base, which was
section. The temperature of the water bath was set to 25 ↓ C.
heated to 150 ↓ C and poured into a mould (for details see [25]
Images were reconstructed using the delay-and-sum
and [27]). Different concentrations of titanium dioxide (TiO2 ,
algorithm with the PATATO toolbox [32] with a bandpass filter
Sigma Aldrich 232033-100g) and alcohol-soluble nigrosin
between 5 KHz and 7 MHz, application of a Hilbert transform,
(Sigma Aldrich 211680-100G) were used to tune µ↔s and µa
as well as time interpolation by a factor of three and detection
respectively. Each phantom consisted of a background cylinder
element interpolation by a factor of two. The images were
into which imaging targets (referred to as inclusions) were
reconstructed into a 300 ↗ 300 pixel grid with a field of
added. The background shape was created using the cut body
view of 32 ↗ 32 mm. To enable compatibility with the U-Net,
from a glass syringe (Sigma Aldrich CADG5157). Addition
images were cropped to 288 ↗ 288 pixels without changing
of the inclusions was performed using either careful pouring
the resolution (Fig. 1 A).
(diameter > 6 mm) or via a vacuum pump. Space for the inclu-
sions was created by either using metal insertion rods with
varying diameters or by using microscopy glass slides to shield D. Data Quality Assurance
an area of the glass mould. Inclusions were prepared in such A wide range of artefacts can corrupt the phantom prepa-
a way as to be z-invariant. Optical properties were sampled ration and imaging process when creating a wide range of
from the ranges µa = 0.05 cm→1 – 0.5 cm→1 in the background inclusion geometries. Sources of artefacts include: unsolved
1216 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 43, NO. 3, MARCH 2024
H. Performance Evaluation
For the final performance evaluation, the estimates of each
of the five training folds were averaged. In addition to per-
formance evaluation on the held-out test phantoms, we also
applied the network to completely different experimental data
without additional training. To this end, we used a blood
flow phantom experiment and in vivo imaging of healthy
mice. While these experiments allow us to investigate the
generalisability and limitations of the models, inaccuracies
are expected due to variability in the Grüneisen parameter
distribution compared to the phantoms.
1) Flow Phantom Data: A variable oxygenation blood flow
phantom was imaged as described previously [38]. We fabri-
cated agar-based cylindrical phantoms with a radius of 10 mm
and embedded a polyvinyl chloride tube (i.d. 0.8 mm, o.d.
0.9 mm) in the centre of the phantom during the agar setting
process at room temperature. For the base mixture, 1.5 % (w/v)
agar (Sigma Aldrich 9012-36-6) was added to Milli-Q water
and was heated until it dissolved. Once the mixture had cooled
to approximately 40↓ C, 2.08% (v/v) of pre-warmed intralipid
(20% emulsion, Merck, 68890-65-3) and 0.74% (v/v) Nigrosin
solution (0.5 mg/ml in Milli-Q water) was added, mixed, and
Fig. 2. Optical absorption estimation methods. (A) The baseline poured into the mould. PA images of the phantom were taken
model (Cal.) is a linear mapping from the arbitrary units PA image to between 700 nm and 900 nm in 20 nm steps.
reference absorption units. (B) Adding complexity, the simulated light
fluence is used to correct PA images (GT-!) before a linear mapping is
2) In Vivo Mouse Data: All animal procedures were con-
applied. (C) For supervised deep learning, a U-Net is trained on simulated ducted in accordance with project and personal licences (PPL
input data to estimate the optical absorption based on the input PAI data number: PE12C2B96), issued under the United Kingdom Ani-
(DL-Sim) and (D) trained on experimentally acquired data (DL-Exp).
mals (Scientific Procedures) Act, 1986 and approved locally
under compliance form CFSB2317. Nine healthy 9-week-
old female C57BL/6 albino mice were imaged according to
a previously established standard operating procedure [31].
To make use of the digital twin simulations during training, Imaging was performed at 10 wavelengths between 700 and
we defined the U-Net to have a two-channel output: the first 900 nm averaging over 10 scans each. The kidneys, spleen,
channel to estimate µa and the second channel to estimate ϖ. spine, and aorta were segmented in PA images using MITK.
Our U-Net implementation used strided convolutions with a 3) Performance Measures: In this work, we report the rela-
(3 ↗ 3) kernel size for downsampling and strided transpose tive error, which is defined as Rel. Error(x, x̂) = |x→x x̂| ↘ 100,
convolutions with a (2 ↗ 2) kernel size for upsampling. It used and the absolute error, which is defined as Abs. Error(x, x̂) =
leaky rectified linear units as the non-linear activation layers |x → x̂|, where x̂ is the estimate for a quantity of interest
and has four sampling steps with a total of 1024 channels in x. We compute the mean of the pixel-wise measure for
the bottleneck and used the mean squared error as the loss each segmentation region. When reporting aggregate results,
function for both output channels. we proceed to compute the median and interquartile range
1218 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 43, NO. 3, MARCH 2024
Fig. 3. Representative performance examples from the held-out test set. (A, F, K) Baseline model (Cal., yellow); (B, G, L) Fluence correction
with reference simulations (GT-ω, green); (C, H, M) U-Net trained on simulated data (DL-Sim, blue); and (D, I, N) U-Net trained on experimental data
(DL-Exp, red). (E, J, O) show line profiles with the respective colours. The methods are colour coded in their dedicated colours and the ground truth
optical absorption is shown in a solid black line (GT µa ). The level windows are adjusted to clearly visualise the water background, the background
material, and the inclusions.
TABLE I
C OMPARISON OF P ERFORMANCE OF THE D IFFERENT M ETHODS .
VALUES A RE S HOWN AS M EDIAN ± I NTERQUARTILE R ANGE /2. R =
P EARSON C ORRELATION C OEFFICIENT ; G CNR = G ENERALISED
C ONTRAST- TO -N OISE R ATIO
IV. D ISCUSSION
We developed a supervised deep learning method to address
the challenge of estimating optical absorption coefficients in
PAI enabled by a collection of carefully annotated phantoms.
The dataset consists of an unprecedented 102 tissue-mimicking
phantoms spanning an important biological range of tissue
optical properties. When assuming independence of images at
21 captured wavelengths from 700 nm to 900 nm, our dataset
features a total of 2142 unique PA images of targets with
distinct and known µa and µ↔s distributions. Each image
was subject to manual segmentation and quality assessment,
then assigned reference optical absorption and scattering coef-
ficients as determined by independent DIS measurements.
Fig. 6. Absorption estimation methods applied in living subjects. We used digital twins of the phantoms to simulate a synthetic
(A) Calibrated PA signal (Cal.) and (B) corresponding oxygenation (sO2 ).
C) µa estimates of DL-Sim and (D) corresponding sO2 . E) µa estimates replica data set to analyse the differences between using
of DL-Exp and (F) corresponding sO2 . G shows the relative estimated simulated and synthetic data sets for model training.
µa ratios between the aorta and spinal region over nine subjects and
H shows the distribution of sO2 values in the aorta. The expected sO2
of 96%±2% is shown as a green area. A. Discussion of Results
We show that erroneous model assumptions have an influ-
ence on the quantitative accuracy of the deep learning model
D. Oxygenation Maps Derived From Learned µa
and that training with experimentally acquired phantoms
Estimates Show More Realistic Arterial Oxygen
has advantages in both accuracy and generalisability of the
Saturation
approach. Our results indicate that tissue-mimicking phantoms
Finally, we tasked Cal., DL-Sim, and DL-Exp methods to can be used as reliable calibration objects and that using
estimate µa in vivo (Fig. 6A,C,E) and subsequently calculated experimental training data is beneficial compared to synthetic
sO2 from multispectral estimates (Fig. 6B,D,F). Despite their training data, indicating the existence of a simulation gap.
1222 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 43, NO. 3, MARCH 2024
At the same time, we confirm that linear calibration schemes experimental and simulated data are expected to be sensitive
fail as they are unable to account for the exponential decay to changes in ϑ of the imaged material, which could be reasons
of fluence with depth. We find that using the experimental for the remaining discrepancies in the flow phantom data set.
data set yields improved accuracy across multiple test cases Similarly for the in vivo application in a mouse, DL-Exp gave a
compared to simulated data sets, at least given the simulation much more realistic representation of the absorption difference
pipeline used in this work. between the aorta and spine, and resulted in a substantial
The deep learning models were initially applied to a test elevation of the estimated arterial oxygen saturation. The
phantom set and results were compared with the naïve cali- final results remained an under-estimate compared to the
bration model and fluence compensation using Monte Carlo expected ⇐ 94% but suggest a greater resilience to depth-
modelling (assuming known optical properties). All methods dependent attenuation.
could recover some correlation of the estimated µa coefficients The U-Nets produced estimates with sharp edges on
compared to the reference µa . A striking result of DL-Exp is the blood-background interface in the flow phantom, likely
the overall improvement in spatial accuracy of the predictions, because of the consistent nature of the spatial frequency
which is demonstrated by smooth estimates with sharply content of the images. Contrary to the promising spectral
defined borders between segmentation classes, quantified by performance, when applied to the mouse data, the U-Nets
consistently high gCNR values. On the other hand, we found were not able to recover the high spatial frequency content
a median relative quantification error of ↑ 29% when training of the image, suggesting a need to enrich the training data set
on experimentally acquired data and ↑ 37% when training on with inclusions that better reflect the complexity of the in vivo
simulated data. In comparison, fluence correction with a Monte imaging application in order to achieve high spatial accuracy
Carlo model using the reference absorption and scattering in the general case.
measurements (GT-ϖ), achieves a relative absorption estima-
tion error of ↑ 22%, but is less accurate in terms of spatial B. Experimental Limitations
estimation at depth and not applicable to settings without While training on experimental phantom measurements
known optical tissue properties. Furthermore, GT-ϖ systemati- shows promise to advance quantitative µa estimates in PAI,
cally overestimates the absorption in depth, indicating a model there are also limitations that must be addressed in future
mismatch in the optical or acoustic forward models. DL-Sim work. Considering first experimental limitations, we only use
systematically overestimates the background absorption at the titanium dioxide to tune µ↔s and nigrosin to tune µa in the
upper side of the phantoms, which could either indicate a phantoms. This is important to minimise the influence of the
mismatch in the light model, or an over/underestimation of molecule- and concentration-dependent Grüneisen parameter
the limited view artefact in the forward model compared to but might introduce a wavelength-dependent bias to the train-
the experimental data as well. ing data that may challenge our assumption of independence
When evaluating using only imaging targets with µa ≃ of images across wavelengths. We assumed images from
2.5 cm→1 , we find that DL-Exp begins to outperform GT-ϖ on different wavelengths to be independent, to maximise the
several reported measures. By extending the training data set, number of training images even though images at neighbouring
we hope to achieve significantly better quantification results. wavelengths will have a high correlation. While such an
Improvement of the deep learning model to mimic traditional assumption could potentially lead to the introduction of a
iterative schemes might also be a promising way forward to wavelength-dependent bias, our results do not show an indi-
further improve quantitative performance. cation for it, suggesting that the assumption – while not being
When moving to completely different phantom and exper- ideal – did not systematically affect the learned algorithms.
imental data, the U-Nets performed less well in terms of If the assumption does not hold, it could impact the quan-
spatial predictions. The significant difference in the spatial tification of multispectrally derived biomarkers, such as blood
distributions of chromophores between the training and test oxygenation sO2 .
data was made intentionally and highlights the limited gen- A second experimental limitation of the study is the
eralisability of the method to data far outside of the training assumption that DIS measurements can be taken as a ground
data distribution. For example, the µa estimate of blood in truth [46], where in fact they may introduce substantial uncer-
the flow phantom was significantly increased although it was tainty [47] for example from modelling errors by a priori
still nearly a factor of three too low compared to the expected assumptions for the sphere parameters, sphere corrections,
µa from literature. We did not include 3D acoustic forward or measurement error for sample thicknesses. Accurate quan-
modelling with consideration of the acoustic attenuation of tification of optical properties is highly challenging, with
the medium and the angular and frequency response of the variation of up to 30% and 40% for µa and µ↔s , respectively,
transducers or the Grüneisen parameter ϑ in our numerical when quantifying the same sample with different instru-
models. We did not conduct measurements for estimations of ments [48]. We examined the influence of our uncertainties and
ϑ for our fabricated materials, however, the large potential found a standard error of 2.5% in the thickness measurements,
discrepancy of ϑ between blood (0.152-0.226) [43], water which translates to approx. 5% error of the optical absorp-
(↑ 0.11) [44], and mineral oil (↑ 0.8) [45] could be one tion coefficient and simulated initial pressure distribution.
explanation for the low µa estimates on the flow phantom In conjunction with the 10%-20% error we determined for
data. While the acoustic modelling aspect is only relevant the DIS system based on cross-reference with time-resolved
when using simulated training data, models trained on both near-infrared spectroscopy [27], our reference optical material
GRÖHL et al.: MOVING BEYOND SIMULATION: DATA-DRIVEN QUANTITATIVE PHOTOACOUSTIC IMAGING 1223
properties are subject to a standard error in the order of 20%. computational modelling of the forward processes involved
In future work, these uncertainties could be incorporated into in PAI, we demonstrate that our phantoms can be used
model training by using Bayesian deep learning methods. as reliable calibration objects. Our experiments confirm the
limitations of linear calibration schemes in accounting for
C. Computational Limitations
depth-dependent fluence effects and show that a deep learn-
We introduced a number of approximations during the ing method exclusively trained on experimentally acquired
Monte Carlo and k-space simulations, such as 2D acoustic datasets can provide better optical property estimation of
modelling (motivated by the out-of-plane symmetry of our previously unseen test phantoms compared to simulated data
phantoms), the laser source geometry and beam profile, and sets. With demonstrations in different test scenarios, including
absolute homogeneity of the phantom materials (i.e. sound in living subjects, deep learning shows potential for real-world
speed and density). We used the delay-and-sum algorithm as applications. Despite remaining limitations in spectral and
the acoustic inverse operator, which is the most commonly spatial accuracy, our work represents an exciting advancement
used reconstruction algorithm even though it is known to towards the development of accurate and reliable quantitative
introduce artefacts to the reconstructed images for our imaging PAI data analysis.
setup [49]. We assume that a combination of these modelling
errors in conjunction with the presence of measurement noise
deep inside tissue is the reason for the relatively high µa quan- D ISCLOSURES
tification error for GT-ϖ. The influence of noise is highlighted For the purpose of open access, the authors have applied a
when visualising the absorption estimated within larger inclu- Creative Commons Attribution (CC BY) licence to any Author
sions with high µa , leading to an increasing overestimation Accepted Manuscript version arising from this submission.
of µa with depth and a wavelength-dependent performance The data to reproduce the findings of this study will be
of GT-ϖ. Using a network architecture that better mirrors the made available via https://doi.org/10.17863/CAM.96644
QPAI problem might increase the quantification performance, upon publication of this manuscript. The respective
either by using a multi-path network with dedicated loss code work is available on GitHub at https://github.com/
functions [24], or by including a model into the network and BohndiekLab/end_to_end_phantom_QPAT.
learning the update for an iterative optimisation scheme [15].
D. Future Directions R EFERENCES
When only analysing results for absorbers with µa ≃ [1] S. Manohar and M. Dantuma, “Current and future trends in photoacous-
2.5 cm→1 , our findings suggest that a learning-based approach tic breast imaging,” Photoacoustics, vol. 16, Dec. 2019, Art. no. 100134.
[2] X. L. Deán-Ben, A. Ozbek, and D. Razansky, “Volumetric real-time
has distinct advantages that might allow it to eventually outper- tracking of peripheral human vasculature with GPU-accelerated three-
form fluence correction. Learning-based models might learn to dimensional optoacoustic tomography,” IEEE Trans. Med. Imag., vol. 32,
correct for effects such as measurement noise, out-of-plane no. 11, pp. 2050–2055, Nov. 2013.
[3] S. Y. Chuah et al., “Structural and functional 3D mapping of skin
signal contributions or signal blurring caused by the point tumours with non-invasive multispectral optoacoustic tomography,” Skin
spread function of the measurement system. Unfortunately, our Res. Technol., vol. 23, no. 2, pp. 221–226, May 2017.
results also show that the proposed approach is far from being [4] J. Aguirre et al., “Precision assessment of label-free psoriasis biomarkers
with ultra-broadband optoacoustic mesoscopy,” Nature Biomed. Eng.,
usable in a clinical setting. To achieve this, we require: (1) vol. 1, no. 5, pp. 1–8, May 2017.
more and broader training data with a focus on more realistic [5] F. Knieling et al., “Multispectral optoacoustic tomography for assess-
spatial chromophore distributions; (2) highly optimised and ment of Crohn’s disease activity,” New England J. Med., vol. 376, no. 13,
accurate optical and 3D acoustic forward models; (3) realistic pp. 1292–1294, Mar. 2017.
[6] M. Xu and L. V. Wang, “Photoacoustic imaging in biomedicine,” Rev.
noise models; as well as (4) model-based acoustic inversion Sci. Instrum., vol. 77, no. 4, Apr. 2006, Art. no. 041101.
schemes to obtain more faithful reconstructions of the initial [7] P. Beard, “Biomedical photoacoustic imaging,” Interface Focus, vol. 1,
pressure distributions. Using a domain transfer approach, such no. 4, pp. 602–631, Aug. 2011.
[8] H. Jaffe and D. A. Berlincourt, “Piezoelectric transducer materials,”
as used by Li et al. [24] or Dreher et al. [22] to complement Proc. IEEE, vol. 53, no. 10, pp. 1372–1386, Oct. 1965.
the experimental data set used in this work with additional [9] J. M. Vaughan, The Fabry–Perot Interferometer: History, Theory, Prac-
simulated data might also be a promising direction. Finally, tice and Applications. Evanston, IL, USA: Routledge, 2017.
[10] B. Cox, J. Laufer, and P. Beard, “The challenges for quantitative pho-
in order to validate the trained methods, diverse in vivo toacoustic imaging,” in Proc. SPIE, vol. 7177, pp. 294–302, Feb. 2009.
test data sets have to be acquired that are representative of [11] B. Cox, J. G. Laufer, S. R. Arridge, and P. C. Beard, “Quantitative spec-
the target applications and annotated with high-quality and troscopic photoacoustic imaging: A review,” J. Biomed. Opt., vol. 17,
no. 6, 2012, Art. no. 061202.
high-confidence labels of the underlying physiology. Since [12] B. T. Cox, S. R. Arridge, K. P. Köstli, and P. C. Beard, “Two-dimensional
ground truth knowledge of the underlying optical properties quantitative photoacoustic image reconstruction of absorption distribu-
in vivo is rarely available, the validation of learned algorithms tions in scattering media by use of a simple iterative method,” Appl.
Opt., vol. 45, no. 8, pp. 1866–1875, 2006.
remains a major roadblock on the path towards quantitative [13] T. Tarvainen, A. Pulkkinen, B. T. Cox, J. P. Kaipio, and S. R. Arridge,
photoacoustics. “Bayesian image reconstruction in quantitative photoacoustic tomog-
raphy,” IEEE Trans. Med. Imag., vol. 32, no. 12, pp. 2287–2298,
V. C ONCLUSION Dec. 2013.
We introduce a collection of well-characterised phantoms to [14] J. Buchmann, B. Kaplan, S. Powell, S. Prohaska, and J. Laufer, “Quan-
titative PA tomography of high resolution 3-D images: Experimental
enable supervised training and evaluation of learned quantita- validation in a tissue phantom,” Photoacoustics, vol. 17, Mar. 2020,
tive PAI methods. Leveraging the power of deep learning and Art. no. 100157.
1224 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 43, NO. 3, MARCH 2024
[15] A. Hauptmann et al., “Model-based learning for accelerated, limited- [33] I. Wolf et al., “The medical imaging interaction toolkit (MITK): A toolkit
view 3-D photoacoustic tomography,” IEEE Trans. Med. Imag., vol. 37, facilitating the creation of interactive software by extending VTK and
no. 6, pp. 1382–1393, Jun. 2018. ITK,” Proc. SPIE, vol. 5367, pp. 16–27, May 2004.
[16] S. Antholzer, M. Haltmeier, and J. Schwab, “Deep learning for pho- [34] Q. Fang and D. A. Boas, “Monte Carlo simulation of photon migration
toacoustic tomography from sparse data,” Inverse Problems Sci. Eng., in 3D turbid media accelerated by graphics processing units,” Opt. Exp.,
vol. 27, no. 7, pp. 987–1005, Jul. 2019. vol. 17, no. 22, pp. 20178–20190, 2009.
[17] C. Cai, K. Deng, C. Ma, and J. Luo, “End-to-end deep neural network [35] B. E. Treeby and B. T. Cox, “k-wave: MATLAB toolbox for the
for optical inversion in quantitative photoacoustic imaging,” Opt. Lett., simulation and reconstruction of photoacoustic wave fields,” Proc. SPIE,
vol. 43, no. 12, pp. 2752–2755, Jun. 2018. vol. 15, no. 2, 2010, Art. no. 021314.
[18] J. Gröhl, T. Kirchner, T. Adler, and L. Maier-Hein, “Confidence estima- [36] G. M. Hale and M. R. Querry, “Optical constants of water in the 200-nm
tion for machine learning-based quantitative photoacoustics,” J. Imag., to 200-µm wavelength region,” Appl. Opt., vol. 12, no. 3, pp. 555–563,
vol. 4, no. 12, p. 147, Dec. 2018. 1973.
[19] T. Chen, T. Lu, S. Song, S. Miao, F. Gao, and J. Li, “A deep learning [37] J. Gröhl et al., “SIMPA: An open-source toolkit for simulation and image
method based on U-Net for quantitative photoacoustic imaging,” Proc. processing for photonics and acoustics,” Proc. SPIE, vol. 27, no. 8,
SPIE, vol. 11240, pp. 216–223, Feb. 2020. Apr. 2022, Art. no. 083010.
[20] J. Gröhl, M. Schellenberg, K. Dreher, and L. Maier-Hein, “Deep learn- [38] M. Gehrung, S. E. Bohndiek, and J. Brunker, “Development of a blood
ing for biomedical photoacoustic imaging: A review,” Photoacoustics, oxygenation phantom for photoacoustic tomography combined with
vol. 22, Jun. 2021, Art. no. 100241. online pO2 detection and flow spectrometry,” Proc. SPIE, vol. 24, no. 12,
[21] M. Schellenberg et al., “Photoacoustic image synthesis with gen- 2019, Art. no. 121908.
erative adversarial networks,” Photoacoustics, vol. 28, Dec. 2022, [39] A. Rodriguez-Molares et al., “The generalized contrast-to-noise ratio:
Art. no. 100402. A formal definition for lesion detectability,” IEEE Trans. Ultrason.,
[22] K. K. Dreher et al., “Unsupervised domain transfer with conditional Ferroelectr., Freq. Control, vol. 67, no. 4, pp. 745–759, Apr. 2020.
invertible neural networks,” 2023, arXiv:2303.10191. [40] S. L. Jacques, “Optical properties of biological tissues: A review,” Phys.
[23] J. Gröhl et al., “Learned spectral decoloring enables photoacoustic Med. Biol., vol. 58, no. 11, pp. R37–R61, Jun. 2013.
oximetry,” Sci. Rep., vol. 11, no. 1, p. 6565, Mar. 2021. [41] L. C. Henyey and J. L. Greenstein, “Diffuse radiation in the galaxy,”
[24] J. Li et al., “Deep learning-based quantitative optoacoustic tomography Astrophys. J., vol. 93, pp. 70–83, Jan. 1941.
of deep tissues in the absence of labeled experimental data,” Optica, [42] A. M. Loeven, C. N. Receno, C. M. Cunningham, and L. R. DeRuisseau,
vol. 9, no. 1, pp. 32–41, 2022. “Arterial blood sampling in male CD-1 and C57BL/6J mice with 1%
[25] L. Hacker et al., “A copolymer-in-oil tissue-mimicking material with isoflurane is similar to awake mice,” J. Appl. Physiol., vol. 125, no. 6,
tuneable acoustic and optical characteristics for photoacoustic imaging pp. 1749–1759, Dec. 2018.
phantoms,” IEEE Trans. Med. Imag., vol. 40, no. 12, pp. 3593–3603, [43] D.-K. Yao, C. Zhang, K. Maslov, and L. V. Wang, “Photoacoustic
Dec. 2021. measurement of the Grüneisen parameter of tissue,” Proc. SPIE, vol. 19,
[26] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional net- no. 1, 2014, Art. no. 017007.
works for biomedical image segmentation,” in Proc. Int. Conf. Med. [44] E. Petrova, S. Ermilov, R. Su, V. Nadvoretskiy, A. Conjusteau, and
Image Comput. Comput.-Assist. Intervent. Cham, Switzerland: Springer, A. Oraevsky, “Using optoacoustic imaging for measuring the tem-
2015, pp. 234–241. perature dependence of Grüneisen parameter in optically absorbing
[27] L. Hacker, “Validation of photoacoustic imaging biomarkers in cancer solutions,” Opt. Exp., vol. 21, no. 21, pp. 25077–25090, 2013.
biology,” Ph.D. dissertation, Univ. Cambridge, Cambridge, U.K., 2022. [45] S. Liang, B. Lashkari, S. S. S. Choi, V. Ntziachristos, and A.
[28] J. W. Pickering, S. A. Prahl, N. Van Wieringen, J. F. Beek, Mandelis, “The application of frequency-domain photoacoustics to
H. J. Sterenborg, and M. J. Van Gemert, “Double-integrating-sphere temperature-dependent measurements of the Grüneisen parameter in
system for measuring the optical properties of tissue,” Appl. Opt., lipids,” Photoacoustics, vol. 11, pp. 56–64, Sep. 2018.
vol. 32, no. 4, pp. 399–410, 1993. [46] E. Zamora-Rojas, B. Aernouts, A. Garrido-Varo, D. Pérez-Marín,
[29] S. A. Prahl, “Everything I think you should know about inverse adding- J. E. Guerrero-Ginel, and W. Saeys, “Double integrating sphere mea-
doubling,” Oregon Med. Laser Center, St. Vincent Hospital, Portland, surements for estimating optical properties of pig subcutaneous adipose
OR, USA, Tech. Rep., 2011, pp. 1–74. tissue,” Innov. Food Sci. Emerg. Technol., vol. 19, pp. 218–226,
[30] C. J. M. Jones and P. R. T. Munro, “Stability of gel wax based optical Jul. 2013.
scattering phantoms,” Biomed. Opt. Exp., vol. 9, no. 8, pp. 3495–3502, [47] J. F. Beek et al., “In vitro double-integrating-sphere optical properties
Aug. 2018. of tissues between 630 and 1064 nm,” Phys. Med. Biol., vol. 42, no. 11,
[31] J. Joseph, M. R. Tomaszewski, I. Quiros-Gonzalez, J. Weber, J. Brunker, pp. 2255–2261, Nov. 1997.
and S. E. Bohndiek, “Evaluation of precision in optoacoustic tomogra- [48] A. Pifferi et al., “Performance assessment of photon migration instru-
phy for preclinical imaging in living subjects,” J. Nucl. Med., vol. 58, ments: The MEDPHOT protocol,” Appl. Opt., vol. 44, no. 11,
no. 5, pp. 807–814, May 2017. pp. 2104–2114, 2005.
[32] T. Else, J. Gröhl, L. Hacker, and S. E. Bohndiek, “PATATO: A Python [49] M. Xu and L. V. Wang, “Universal back-projection algorithm for
photoacoustic tomography analysis toolkit,” Proc. SPIE, vol. PC12379, photoacoustic computed tomography,” Phys. Rev. E, Stat. Phys. Plasmas
Mar. 2023, Art. no. PC123790T. Fluids Relat. Interdiscip. Top., vol. 71, no. 1, Jan. 2005, Art. no. 016706.